[jira] [Resolved] (HBASE-16641) QA tests for hbase-client skip the second part.
[
https://issues.apache.org/jira/browse/HBASE-16641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-16641.
---
Resolution: Duplicate
> QA tests for hbase-client skip the second part.
> ---
>
> Key: HBASE-16641
> URL: https://issues.apache.org/jira/browse/HBASE-16641
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>
> See
> https://builds.apache.org/job/PreCommit-HBASE-Build/3547/artifact/patchprocess/patch-unit-hbase-client.txt
> {code}
> [INFO] --- maven-surefire-plugin:2.18.1:test (secondPartTestsExecution) @
> hbase-client ---
> [INFO] Tests are skipped.
> {code}
> The first part passed fine, but second parts is skipped.
> Notice hbase-client/pom.xml
> {code}
>
>
> secondPartTestsExecution
> test
>
> test
>
>
> true
>
>
>
> {code}
> If i change the 'skip' to be false, the second part could be triggered. But
> this configuration existed for a long time, is the cmd line on build box
> updated recently?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16702) TestBlockEvictionFromClient is broken
Heng Chen created HBASE-16702: - Summary: TestBlockEvictionFromClient is broken Key: HBASE-16702 URL: https://issues.apache.org/jira/browse/HBASE-16702 Project: HBase Issue Type: Bug Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16665) Check whether KeyValueUtil.createXXX could be replaced by CellUtil without copy
Heng Chen created HBASE-16665: - Summary: Check whether KeyValueUtil.createXXX could be replaced by CellUtil without copy Key: HBASE-16665 URL: https://issues.apache.org/jira/browse/HBASE-16665 Project: HBase Issue Type: Bug Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16652) Figure out performance difference between increment and append
Heng Chen created HBASE-16652: - Summary: Figure out performance difference between increment and append Key: HBASE-16652 URL: https://issues.apache.org/jira/browse/HBASE-16652 Project: HBase Issue Type: Sub-task Reporter: Heng Chen When do performance test in HBASE-16625, i found it has the very big difference between Append and Increment (append is about 37% faster than increment). As [~stack] mentioned in https://issues.apache.org/jira/browse/HBASE-16610?focusedCommentId=15493166=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15493166, append and increment has been unified in server-side, and they looks the same in client-side. This issue is to figure out why the performance looks different between them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16641) QA tests for hbase-client skip the second part.
Heng Chen created HBASE-16641: - Summary: QA tests for hbase-client skip the second part. Key: HBASE-16641 URL: https://issues.apache.org/jira/browse/HBASE-16641 Project: HBase Issue Type: Bug Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16631) Extract AsyncRequestFuture relates code from AsyncProcess
Heng Chen created HBASE-16631: - Summary: Extract AsyncRequestFuture relates code from AsyncProcess Key: HBASE-16631 URL: https://issues.apache.org/jira/browse/HBASE-16631 Project: HBase Issue Type: Bug Reporter: Heng Chen Assignee: Heng Chen Now, AsyncProcess class is too large (over 2000+ lines), and there are so many sub classes in it. AsyncRequestFutureImpl is the most biggest subclass in AP, we could extract it out from AP to reduce the AP size. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16625) Performance test for interface unified with AP
Heng Chen created HBASE-16625: - Summary: Performance test for interface unified with AP Key: HBASE-16625 URL: https://issues.apache.org/jira/browse/HBASE-16625 Project: HBase Issue Type: Sub-task Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16623) Unify Get with AP
Heng Chen created HBASE-16623: - Summary: Unify Get with AP Key: HBASE-16623 URL: https://issues.apache.org/jira/browse/HBASE-16623 Project: HBase Issue Type: Sub-task Reporter: Heng Chen Assignee: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16611) Flakey org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet
Heng Chen created HBASE-16611:
-
Summary: Flakey
org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet
Key: HBASE-16611
URL: https://issues.apache.org/jira/browse/HBASE-16611
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
see
https://builds.apache.org/job/PreCommit-HBASE-Build/3494/artifact/patchprocess/patch-unit-hbase-server.txt
{code}
testCancelOfMultiGet(org.apache.hadoop.hbase.client.TestReplicasClient) Time
elapsed: 4.026 sec <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet(TestReplicasClient.java:579)
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 94.401 sec -
in org.apache.hadoop.hbase.client.TestAdmin2
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.861 sec - in
org.apache.hadoop.hbase.client.TestClientScannerRPCTimeout
Running
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClientWithRegionReplicas
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 261.925 sec <<<
FAILURE! - in org.apache.hadoop.hbase.client.TestReplicasClient
testCancelOfMultiGet(org.apache.hadoop.hbase.client.TestReplicasClient) Time
elapsed: 4.522 sec <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet(TestReplicasClient.java:581)
Running org.apache.hadoop.hbase.client.TestFastFail
Tests run: 2, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 3.648 sec - in
org.apache.hadoop.hbase.client.TestFastFail
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 277.894 sec <<<
FAILURE! - in org.apache.hadoop.hbase.client.TestReplicasClient
testCancelOfMultiGet(org.apache.hadoop.hbase.client.TestReplicasClient) Time
elapsed: 5.359 sec <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.hbase.client.TestReplicasClient.testCancelOfMultiGet(TestReplicasClient.java:579)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16610) Unify append, increment with AP
Heng Chen created HBASE-16610: - Summary: Unify append, increment with AP Key: HBASE-16610 URL: https://issues.apache.org/jira/browse/HBASE-16610 Project: HBase Issue Type: Sub-task Reporter: Heng Chen Assignee: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16607) Make NoncedRegionServerCallable extends CancellableRegionServerCallable
Heng Chen created HBASE-16607: - Summary: Make NoncedRegionServerCallable extends CancellableRegionServerCallable Key: HBASE-16607 URL: https://issues.apache.org/jira/browse/HBASE-16607 Project: HBase Issue Type: Sub-task Reporter: Heng Chen This is the first step to unify append, increment with AP. And after extends CancellableRegionServerCallable, we could remove lots of duplicate code in NoncedRegionServerCallable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16606) Remove some duplicate code in HTable
Heng Chen created HBASE-16606: - Summary: Remove some duplicate code in HTable Key: HBASE-16606 URL: https://issues.apache.org/jira/browse/HBASE-16606 Project: HBase Issue Type: Sub-task Reporter: Heng Chen Assignee: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16597) Revisit the threadPool is really needed in submitAll and submit interface in AsyncProcess
[ https://issues.apache.org/jira/browse/HBASE-16597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen resolved HBASE-16597. --- Resolution: Invalid > Revisit the threadPool is really needed in submitAll and submit interface in > AsyncProcess > - > > Key: HBASE-16597 > URL: https://issues.apache.org/jira/browse/HBASE-16597 > Project: HBase > Issue Type: Sub-task >Reporter: Heng Chen >Assignee: Heng Chen > > Currently, threadPool could be passed into AP by constructor and submitAll, > submit interfaces, but i notice in HTable the pool passed to AP though > submitAll looks same with the one in AP as default, Let me revisit it to > ensure whether the pool is really needed in submitAll and submit interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16597) Revisit the threadPool is really needed in submitAll and submit interface in AsyncProcess
Heng Chen created HBASE-16597: - Summary: Revisit the threadPool is really needed in submitAll and submit interface in AsyncProcess Key: HBASE-16597 URL: https://issues.apache.org/jira/browse/HBASE-16597 Project: HBase Issue Type: Sub-task Reporter: Heng Chen Currently, threadPool could be passed into AP by constructor and submitAll, submit interfaces, but i notice in HTable the pool passed to AP though submitAll looks same with the one in AP as default, Let me revisit it to ensure whether the pool is really needed in submitAll and submit interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16596) Reduce redundant interfaces in AP
Heng Chen created HBASE-16596: - Summary: Reduce redundant interfaces in AP Key: HBASE-16596 URL: https://issues.apache.org/jira/browse/HBASE-16596 Project: HBase Issue Type: Sub-task Reporter: Heng Chen Assignee: Heng Chen Currently, there are lots of interfaces in AP, some of them only be used only in test case, or be used only once, let's remove the redundant ones to keep the AP more clear -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16593) Unify HTable with AP
Heng Chen created HBASE-16593: - Summary: Unify HTable with AP Key: HBASE-16593 URL: https://issues.apache.org/jira/browse/HBASE-16593 Project: HBase Issue Type: Umbrella Reporter: Heng Chen Assignee: Heng Chen Currently, HTable has two ways to deal with request, one is call RPC directly, it is used to processed single action request such as Get, Delete, Append, Increment. Another one is through AP to deal with multi action requests, such as batch, mutation etc. This issue is to unify them with AP only. It has some benefits, for example we could implements async interface easily with AP, and we could make the client logic more clear just use AP to communicate with Server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16592) Unify Delete request with AP
Heng Chen created HBASE-16592: - Summary: Unify Delete request with AP Key: HBASE-16592 URL: https://issues.apache.org/jira/browse/HBASE-16592 Project: HBase Issue Type: Task Reporter: Heng Chen Assignee: Heng Chen This is the first step try to unify the HTable with AP only, to extend AP could process single action, i introduced AbstractResponse, multiResponse and singleResponse (introduced to deal with single result) will extend this class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16562) ITBLL should fail to start if misconfigured
[ https://issues.apache.org/jira/browse/HBASE-16562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen resolved HBASE-16562. --- Resolution: Fixed Hadoop Flags: Reviewed > ITBLL should fail to start if misconfigured > --- > > Key: HBASE-16562 > URL: https://issues.apache.org/jira/browse/HBASE-16562 > Project: HBase > Issue Type: Improvement > Components: integration tests >Reporter: Andrew Purtell >Assignee: Heng Chen > Fix For: 2.0.0, 1.0.4, 1.4.0, 1.3.1, 1.1.7, 0.98.23, 1.2.4 > > Attachments: HBASE-16562-branch-1.2.patch, > HBASE-16562-branch-1.2.v1.patch, HBASE-16562.patch, HBASE-16562.v1.patch, > HBASE-16562.v1.patch-addendum > > > The number of nodes in ITBLL must a multiple of width*wrap (defaults to 25M, > but can be configured by adding two more args to the test invocation) or else > verification will fail. This can be very expensive in terms of time or hourly > billing for on demand test resources. Check the sanity of test parameters > before launching any MR jobs and fail fast if invariants aren't met with an > indication what parameter(s) need fixing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16564) ITBLL run failed with hadoop 2.7.2 on branch 0.98
[
https://issues.apache.org/jira/browse/HBASE-16564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-16564.
---
Resolution: Invalid
As [~Apache9] said, the best solution is upgrade hadoop client, so close this
issue as invalid.
> ITBLL run failed with hadoop 2.7.2 on branch 0.98
> -
>
> Key: HBASE-16564
> URL: https://issues.apache.org/jira/browse/HBASE-16564
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Priority: Minor
>
> 0.98 compiled with hadoop 2.2.0, so it has some compatibility issues with
> hadoop 2.7.2 (it seems 2.5.0+ has the same issue), some counter has been
> removed.
> IMO we should catch the exception so our ITBLL could go on.
> {code}
> 16/09/06 15:39:33 INFO hbase.HBaseCluster: Added new HBaseAdmin
> 16/09/06 15:39:33 INFO hbase.HBaseCluster: Restoring cluster - done
> 16/09/06 15:39:33 INFO hbase.HBaseCommonTestingUtility: Stopping mini
> mapreduce cluster...
> 16/09/06 15:39:33 INFO Configuration.deprecation: mapred.job.tracker is
> deprecated. Instead, use mapreduce.jobtracker.address
> 16/09/06 15:39:33 INFO hbase.HBaseCommonTestingUtility: Mini mapreduce
> cluster stopped
> 16/09/06 15:39:33 ERROR util.AbstractHBaseTool: Error running command-line
> tool
> java.lang.IllegalArgumentException: No enum constant
> org.apache.hadoop.mapreduce.JobCounter.MB_MILLIS_MAPS
> at java.lang.Enum.valueOf(Enum.java:238)
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.valueOf(FrameworkCounterGroup.java:148)
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.findCounter(FrameworkCounterGroup.java:182)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
> at
> org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:240)
> at
> org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:370)
> at
> org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:511)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:756)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:753)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:753)
> at
> org.apache.hadoop.mapreduce.Job.monitorAndPrintJob(Job.java:1361)
> at
> org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1289)
> at
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.jobCompletion(IntegrationTestBigLinkedList.java:543)
> at
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.runRandomInputGenerator(IntegrationTestBigLinkedList.java:505)
> at
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.run(IntegrationTestBigLinkedList.java:553)
> at
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.runGenerator(IntegrationTestBigLinkedList.java:842)
> at
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.run(IntegrationTestBigLinkedList.java:892)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList.runTestFromCommandLine(IntegrationTestBigLinkedList.java:1237)
> at
> org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:115)
> at
> org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList.main(IntegrationTestBigLinkedList.java:1272)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16564) ITBLL run failed with hdfs 2.7.2 on branch 0.98
Heng Chen created HBASE-16564:
-
Summary: ITBLL run failed with hdfs 2.7.2 on branch 0.98
Key: HBASE-16564
URL: https://issues.apache.org/jira/browse/HBASE-16564
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
Priority: Minor
0.98 compiled with hdfs 2.2.0, so it has some compatibility issues with hdfs
2.7.2 (it seems 2.5.0+ has the same issue), some counter has been removed.
IMO we should catch the exception so our ITBLL could go on.
{code}
16/09/06 15:39:33 INFO hbase.HBaseCluster: Added new HBaseAdmin
16/09/06 15:39:33 INFO hbase.HBaseCluster: Restoring cluster - done
16/09/06 15:39:33 INFO hbase.HBaseCommonTestingUtility: Stopping mini mapreduce
cluster...
16/09/06 15:39:33 INFO Configuration.deprecation: mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
16/09/06 15:39:33 INFO hbase.HBaseCommonTestingUtility: Mini mapreduce cluster
stopped
16/09/06 15:39:33 ERROR util.AbstractHBaseTool: Error running command-line tool
java.lang.IllegalArgumentException: No enum constant
org.apache.hadoop.mapreduce.JobCounter.MB_MILLIS_MAPS
at java.lang.Enum.valueOf(Enum.java:238)
at
org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.valueOf(FrameworkCounterGroup.java:148)
at
org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.findCounter(FrameworkCounterGroup.java:182)
at
org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
at
org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:240)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:370)
at
org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:511)
at org.apache.hadoop.mapreduce.Job$7.run(Job.java:756)
at org.apache.hadoop.mapreduce.Job$7.run(Job.java:753)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:753)
at org.apache.hadoop.mapreduce.Job.monitorAndPrintJob(Job.java:1361)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1289)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.jobCompletion(IntegrationTestBigLinkedList.java:543)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.runRandomInputGenerator(IntegrationTestBigLinkedList.java:505)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.run(IntegrationTestBigLinkedList.java:553)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.runGenerator(IntegrationTestBigLinkedList.java:842)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.run(IntegrationTestBigLinkedList.java:892)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList.runTestFromCommandLine(IntegrationTestBigLinkedList.java:1237)
at
org.apache.hadoop.hbase.IntegrationTestBase.doWork(IntegrationTestBase.java:115)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:112)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList.main(IntegrationTestBigLinkedList.java:1272)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16512) when locate meta region, we should respect the param "useCache" passed in on 0.98
Heng Chen created HBASE-16512:
-
Summary: when locate meta region, we should respect the param
"useCache" passed in on 0.98
Key: HBASE-16512
URL: https://issues.apache.org/jira/browse/HBASE-16512
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.21
Reporter: Heng Chen
we found that when RS with meta crash, client will retry the same request,
but it still use the original meta location in cache, so all request retried
will failed.
Notice the code in HConnectionMananger#locateRegionInMeta, the "useCache"
passed in is not used when try to found the meta region.
{code}
private HRegionLocation locateRegionInMeta(final TableName parentTable,
final TableName tableName, final byte [] row, boolean useCache,
Object regionLockObject, boolean retry)
throws IOException {
..
for (int tries = 0; true; tries++) {
.
HRegionLocation metaLocation = null;
try {
// locate the meta region
metaLocation = locateRegion(parentTable, metaKey, true, false);
//NOTICE: we should honor the "useCache" passed in when locate the meta region.
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16490) Fix race condition between SnapshotManager and SnapshotCleaner
Heng Chen created HBASE-16490:
-
Summary: Fix race condition between SnapshotManager and
SnapshotCleaner
Key: HBASE-16490
URL: https://issues.apache.org/jira/browse/HBASE-16490
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
Fix For: 2.0.0
As [~mbertozzi] comments on HBASE-16464, there maybe race condition between
SnapshotManager and SnapshotCleaner. We should use one lock when create
snapshot, and cleanup should acquire the lock before take action.
One method is pass HMaster as param into Cleaner through
{{FileCleanerDelegate.getDeletableFiles}}, suggestions are welcomed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16464) archive folder grows bigger and bigger due to corrupt snapshot under tmp dir
Heng Chen created HBASE-16464:
-
Summary: archive folder grows bigger and bigger due to corrupt
snapshot under tmp dir
Key: HBASE-16464
URL: https://issues.apache.org/jira/browse/HBASE-16464
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
We met the problem on our real production cluster, we need to cleanup some
data on hbase, we notice the archive folder is much larger than others, so we
delete all snapshots of all tables, but the archive folder still grows bigger
and bigger.
After check the hmaster log, we notice the exception below:
{code}
2016-08-22 15:34:33,089 ERROR [f04,16000,1471240833208_ChoreService_1]
snapshot.SnapshotHFileCleaner: Exception while checking if files were valid,
keeping them just in case.
org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Couldn't read
snapshot info
from:hdfs://f04/hbase/.hbase-snapshot/.tmp/frog_stastic_2016-08-17/.snapshotinfo
at
org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:295)
at
org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.getHFileNames(SnapshotReferenceUtil.java:328)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner$1.filesUnderSnapshot(SnapshotHFileCleaner.java:85)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache.getSnapshotsInProgress(SnapshotFileCache.java:303)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache.getUnreferencedFiles(SnapshotFileCache.java:194)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner.getDeletableFiles(SnapshotHFileCleaner.java:62)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:233)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:157)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:180)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:149)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:180)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:149)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:180)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:149)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:180)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:149)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:180)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:149)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.chore(CleanerChore.java:124)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: File does not exist:
/hbase/.hbase-snapshot/.tmp/frog_stastic_2016-08-17/.snapshotinfo
at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:71)
at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:61)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1828)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1799)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1712)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:587)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:365)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
[jira] [Created] (HBASE-16427) After HBASE-13701, hbase standalone mode start failed due to mkdir failed
Heng Chen created HBASE-16427:
-
Summary: After HBASE-13701, hbase standalone mode start failed
due to mkdir failed
Key: HBASE-16427
URL: https://issues.apache.org/jira/browse/HBASE-16427
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
{code}
2016-08-17 10:26:43,305 ERROR [main] regionserver.SecureBulkLoadManager: Failed
to create or set permission on staging directory /user/chenheng/hbase-staging
ExitCodeException exitCode=1: chmod: /user/chenheng/hbase-staging: No such file
or directory
at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:815)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:798)
at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:728)
at
org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:502)
at
org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager.start(SecureBulkLoadManager.java:124)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:626)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:406)
at
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:307)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:140)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:221)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:156)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:226)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2421)
2016-08-17 10:26:43,306 ERROR [main] master.HMasterCommandLine: Master exiting
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16258) Some issues about metrices on webUI
Heng Chen created HBASE-16258: - Summary: Some issues about metrices on webUI Key: HBASE-16258 URL: https://issues.apache.org/jira/browse/HBASE-16258 Project: HBase Issue Type: Bug Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16076) Cannot configure split policy in HBase shell
[
https://issues.apache.org/jira/browse/HBASE-16076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-16076.
---
Resolution: Fixed
Assignee: Heng Chen
Commit to master
> Cannot configure split policy in HBase shell
>
>
> Key: HBASE-16076
> URL: https://issues.apache.org/jira/browse/HBASE-16076
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Youngjoon Kim
>Assignee: Heng Chen
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16076.patch, HBASE-16076_v1.patch
>
>
> The reference guide explains how to configure split policy in HBase
> shell([link|http://hbase.apache.org/book.html#_custom_split_policies]).
> {noformat}
> Configuring the Split Policy On a Table Using HBase Shell
> hbase> create 'test', {METHOD => 'table_att', CONFIG => {'SPLIT_POLICY' =>
> 'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy'}},
> {NAME => 'cf1'}
> {noformat}
> But if run that command, shell complains 'An argument ignored (unknown or
> overridden): CONFIG', and the table description has no split policy.
> {noformat}
> hbase(main):067:0* create 'test', {METHOD => 'table_att', CONFIG =>
> {'SPLIT_POLICY' =>
> 'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy'}}, {NAME
> => 'cf1'}
> An argument ignored (unknown or overridden): CONFIG
> Created table test
> Took 1.2180 seconds
> hbase(main):068:0> describe 'test'
> Table test is ENABLED
> test
> COLUMN FAMILIES DESCRIPTION
> {NAME => 'cf1', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
> 'FOREVER', MIN_VERSIONS => '0', IN_MEMORY_COMPACTION => 'false',
> KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => '
> false', BLOCKCACHE => 'true'}
> 1 row(s)
> Took 0.0200 seconds
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-16111) Truncate preserve shell command is broken
[
https://issues.apache.org/jira/browse/HBASE-16111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-16111.
---
Resolution: Fixed
Assignee: Heng Chen
Hadoop Flags: Reviewed
> Truncate preserve shell command is broken
> -
>
> Key: HBASE-16111
> URL: https://issues.apache.org/jira/browse/HBASE-16111
> Project: HBase
> Issue Type: Bug
> Components: shell
>Reporter: Elliott Clark
>Assignee: Heng Chen
> Labels: shell
> Fix For: 2.0.0
>
> Attachments: HBASE-16111.patch
>
>
> On a recent version of master I get this:
> {code}
> hbase(main):001:0> truncate_preserve 'TestTable'
> ERROR: undefined local variable or method `table' for
> #
> Here is some help for this command:
> Disables, drops and recreates the specified table while still maintaing the
> previous region boundaries.
> Took 0.0290 seconds
> hbase(main):002:0> truncate 'TestTable'
> Truncating 'TestTable' table (it may take a while):
> Disabling table...
> Truncating table...
> Took 10.0040 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-15900) RS stuck in get lock of HStore
[
https://issues.apache.org/jira/browse/HBASE-15900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-15900.
---
Resolution: Duplicate
> RS stuck in get lock of HStore
> --
>
> Key: HBASE-15900
> URL: https://issues.apache.org/jira/browse/HBASE-15900
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.1, 1.3.0
>Reporter: Heng Chen
> Attachments: 0d32a6bab354e6cc170cd59a2d485797.jstack.txt,
> 0d32a6bab354e6cc170cd59a2d485797.rs.log, 9fe15a52_9fe15a52_save,
> c91324eb_81194e359707acadee2906ffe36ab130.log, dump.txt
>
>
> It happens on my production cluster when i run MR job. I save the dump.txt
> from this RS webUI.
> Many threads stuck here:
> {code}
> Thread 133 (B.defaultRpcServer.handler=94,queue=4,port=16020):
>32 State: WAITING
>31 Blocked count: 477816
>30 Waited count: 535255
>29 Waiting on
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@6447ba67
>28 Stack:
>27 sun.misc.Unsafe.park(Native Method)
>26 java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>25
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
>24
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
>23
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
>22
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
>21 org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:666)
>20
> org.apache.hadoop.hbase.regionserver.HRegion.applyFamilyMapToMemstore(HRegion.java:3621)
>19
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3038)
>18
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2793)
>17
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2735)
>16
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:692)
>15
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:654)
>14
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2029)
>13
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
>12 org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2112)
>11 org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
>10
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> 9 org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> 8 java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16040) Remove configuration "hbase.replication"
Heng Chen created HBASE-16040: - Summary: Remove configuration "hbase.replication" Key: HBASE-16040 URL: https://issues.apache.org/jira/browse/HBASE-16040 Project: HBase Issue Type: Bug Reporter: Heng Chen Fix For: 2.0.0 This configuration was introduced to reduce overhead of replication. Now the overhead of replication is negligible. Besides that, this config is not in hbase-default.xml, user has to read the code to know about it and its' default value, this is unfriendly. So let's remove it. suggestions? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16031) Documents about "hbase.replication" default value seems wrong
[
https://issues.apache.org/jira/browse/HBASE-16031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-16031.
---
Resolution: Fixed
Assignee: Heng Chen
Fix Version/s: 2.0.0
> Documents about "hbase.replication" default value seems wrong
> -
>
> Key: HBASE-16031
> URL: https://issues.apache.org/jira/browse/HBASE-16031
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0
>
> Attachments: HBASE-16031.patch
>
>
> {code}
> public static final String
> REPLICATION_ENABLE_KEY = "hbase.replication";
> public static final boolean
> REPLICATION_ENABLE_DEFAULT = true;
> {code}
> The code shows that default value is true, but documents shows the default
> value is false.
> {code}
> | hbase.replication
> | Whether replication is enabled or disabled on a given
> cluster
> | false
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16031) Documents about "hbase.replication" default value seems wrong
Heng Chen created HBASE-16031:
-
Summary: Documents about "hbase.replication" default value seems
wrong
Key: HBASE-16031
URL: https://issues.apache.org/jira/browse/HBASE-16031
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
{code}
public static final String
REPLICATION_ENABLE_KEY = "hbase.replication";
public static final boolean
REPLICATION_ENABLE_DEFAULT = true;
{code}
The code shows that default value is true, but documents shows the default
value is false.
{code}
| hbase.replication
| Whether replication is enabled or disabled on a given
cluster
| false
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15900) RS stuck in get lock of HStore
Heng Chen created HBASE-15900:
-
Summary: RS stuck in get lock of HStore
Key: HBASE-15900
URL: https://issues.apache.org/jira/browse/HBASE-15900
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
Attachments: dump.txt
It happens on my production cluster when i run MR job. I save the dump.txt
from this RS webUI.
Many threads stuck here:
{code}
Thread 133 (B.defaultRpcServer.handler=94,queue=4,port=16020):
32 State: WAITING
31 Blocked count: 477816
30 Waited count: 535255
29 Waiting on
java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@6447ba67
28 Stack:
27 sun.misc.Unsafe.park(Native Method)
26 java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
25
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
24
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
23
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
22
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
21 org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:666)
20
org.apache.hadoop.hbase.regionserver.HRegion.applyFamilyMapToMemstore(HRegion.java:3621)
19
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3038)
18
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2793)
17
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2735)
16
org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:692)
15
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:654)
14
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2029)
13
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
12 org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2112)
11 org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
10
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
9 org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
8 java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15844) We should respect hfile.block.index.cacheonwrite when write intermediate index Block
Heng Chen created HBASE-15844:
-
Summary: We should respect hfile.block.index.cacheonwrite when
write intermediate index Block
Key: HBASE-15844
URL: https://issues.apache.org/jira/browse/HBASE-15844
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
{code: title=BlockIndexWriter#writeIntermediateBlock}
if (cacheConf != null) {
HFileBlock blockForCaching = blockWriter.getBlockForCaching(cacheConf);
cacheConf.getBlockCache().cacheBlock(new BlockCacheKey(nameForCaching,
beginOffset, true, blockForCaching.getBlockType()), blockForCaching);
}
{code}
The if condition should be ?
{code}
if (cacheConf != null && cacheConf.shouldCacheIndexesOnWrite())
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15814) Miss important information in Document of HBase Security
Heng Chen created HBASE-15814:
-
Summary: Miss important information in Document of HBase Security
Key: HBASE-15814
URL: https://issues.apache.org/jira/browse/HBASE-15814
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Heng Chen
I have deployed secure cluster recently, and found we miss important
information in http://hbase.apache.org/book.html#security
Some configurations like
{code}
hbase.regionserver.kerberos.principal
hbase/_h...@your-realm.com
hbase.regionserver.keytab.file
/etc/hbase/conf/hbase.keytab
hbase.master.kerberos.principal
hbase/_h...@your-realm.com
hbase.master.keytab.file
/etc/hbase/conf/hbase.keytab
{code}
And i found more detailed document in
http://www.cloudera.com/documentation/enterprise/5-5-x/topics/cdh_sg_hbase_authentication.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15804) Return 404 in some document link
Heng Chen created HBASE-15804:
-
Summary: Return 404 in some document link
Key: HBASE-15804
URL: https://issues.apache.org/jira/browse/HBASE-15804
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
http://hbase.apache.org/book.html#security
The link to {{Understanding User Authentication and Authorization in Apache
HBase} } return 404
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-15720) Print row locks at the debug dump page
[ https://issues.apache.org/jira/browse/HBASE-15720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen resolved HBASE-15720. --- Resolution: Fixed > Print row locks at the debug dump page > -- > > Key: HBASE-15720 > URL: https://issues.apache.org/jira/browse/HBASE-15720 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.1 >Reporter: Enis Soztutar >Assignee: Heng Chen > Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2, 0.98.20, 1.0.5 > > Attachments: 4742C21D-B9CE-4921-9B32-CC319488EC64.png, > HBASE-15720-branch-1.0-addendum.patch, HBASE-15720-branch-1.2-addendum.patch, > HBASE-15720.patch > > > We had to debug cases where some handlers are holding row locks for an > extended time (and maybe leak) and other handlers are getting timeouts for > obtaining row locks. > We should add row lock information at the debug page at the RS UI to be able > to live-debug such cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-15278) AsyncRPCClient hangs if Connection closes before RPC call response
[ https://issues.apache.org/jira/browse/HBASE-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen reopened HBASE-15278: --- revert it. See testcase failed https://builds.apache.org/job/PreCommit-HBASE-Build/1691/testReport/ > AsyncRPCClient hangs if Connection closes before RPC call response > --- > > Key: HBASE-15278 > URL: https://issues.apache.org/jira/browse/HBASE-15278 > Project: HBase > Issue Type: Bug > Components: rpc, test >Affects Versions: 2.0.0 >Reporter: Enis Soztutar >Assignee: Heng Chen >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-15278.patch, HBASE-15278_v1.patch, > HBASE-15278_v2.patch, hbase-15278_v00.patch > > > The test for HBASE-15212 discovered an issue with Async RPC Client. > In that test, we are closing the connection if an RPC call writes a call > larger than max allowed size, the server closes the connection. However the > async client does not seem to handle connection closes with outstanding RPC > calls. The client just hangs. > Marking this blocker against 2.0 since it is default there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15692) We should remove snapshot dir under .tmp when write .snapshotinfo failed.
Heng Chen created HBASE-15692:
-
Summary: We should remove snapshot dir under .tmp when write
.snapshotinfo failed.
Key: HBASE-15692
URL: https://issues.apache.org/jira/browse/HBASE-15692
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
we encounter this problem on our production cluster.
This is exception in HMaster.log
{code}
2016-04-22 11:05:06,390 ERROR [f04,16000,1459941011479_ChoreService_3]
snapshot.SnapshotHFileCleaner: Exception while checking if files were valid,
keeping them just in case.
org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException: Couldn't read
snapshot info
from:hdfs://f04/hbase/.hbase-snapshot/.tmp/frog_stastic_2016-04-07/.snapshotinfo
at
org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils.readSnapshotInfo(SnapshotDescriptionUtils.java:295)
at
org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.getHFileNames(SnapshotReferenceUtil.java:328)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner$1.filesUnderSnapshot(SnapshotHFileCleaner.java:85)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache.getSnapshotsInProgress(SnapshotFileCache.java:303)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotFileCache.getUnreferencedFiles(SnapshotFileCache.java:194)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner.getDeletableFiles(SnapshotHFileCleaner.java:62)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:233)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:157)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:180)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:149)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:180)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:149)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:180)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:149)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:180)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:149)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteDirectory(CleanerChore.java:180)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteEntries(CleanerChore.java:149)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.chore(CleanerChore.java:124)
at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:185)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: File does not exist:
/hbase/.hbase-snapshot/.tmp/frog_stastic_2016-04-07/.snapshotinfo
at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:64)
at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:54)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1795)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1738)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1718)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1690)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:519)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:337)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at
[jira] [Resolved] (HBASE-15642) split region number for one table on webUI never be reduced
[ https://issues.apache.org/jira/browse/HBASE-15642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen resolved HBASE-15642. --- Resolution: Invalid > split region number for one table on webUI never be reduced > > > Key: HBASE-15642 > URL: https://issues.apache.org/jira/browse/HBASE-15642 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.17 >Reporter: Heng Chen > Attachments: 268286F8-8BA4-4144-9B66-1A6F70FEC2EB.png > > > This happened after we upgrade our cluster from 0.98.6 to 0.98.17. > The number should be reduced, but it always increases from original 20+ to > 49 now. (yesterday it was 48) > Need to dig. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15643) Need metrics of cache hit ratio for one table
Heng Chen created HBASE-15643: - Summary: Need metrics of cache hit ratio for one table Key: HBASE-15643 URL: https://issues.apache.org/jira/browse/HBASE-15643 Project: HBase Issue Type: Bug Reporter: Heng Chen There are many tables on our cluster, we only some of them need to be read online. We could improve the performance of read by cache, but we need some metrics for it at table level -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15642) split region number for one table on webUI never be reduced
Heng Chen created HBASE-15642: - Summary: split region number for one table on webUI never be reduced Key: HBASE-15642 URL: https://issues.apache.org/jira/browse/HBASE-15642 Project: HBase Issue Type: Bug Affects Versions: 0.98.17 Reporter: Heng Chen This happened after we upgrade our cluster from 0.98.6 to 0.98.17. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15635) Mean age of Blocks in cache (seconds) on webUI should be greater than zero
Heng Chen created HBASE-15635: - Summary: Mean age of Blocks in cache (seconds) on webUI should be greater than zero Key: HBASE-15635 URL: https://issues.apache.org/jira/browse/HBASE-15635 Project: HBase Issue Type: Bug Affects Versions: 0.98.17 Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15629) Backport HBASE-14703 to 0.98+
Heng Chen created HBASE-15629: - Summary: Backport HBASE-14703 to 0.98+ Key: HBASE-15629 URL: https://issues.apache.org/jira/browse/HBASE-15629 Project: HBase Issue Type: Task Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-15377) Per-RS Get metric is time based, per-region metric is size-based
[
https://issues.apache.org/jira/browse/HBASE-15377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-15377.
---
Resolution: Duplicate
> Per-RS Get metric is time based, per-region metric is size-based
>
>
> Key: HBASE-15377
> URL: https://issues.apache.org/jira/browse/HBASE-15377
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>
> We have metrics for Get operations at the region server level and region
> level.
> {code}
>"Get_num_ops" : 4837505,
> "Get_min" : 0,
> "Get_max" : 296,
> "Get_mean" : 0.2934618155433431,
> "Get_median" : 0.0,
> "Get_75th_percentile" : 0.0,
> "Get_95th_percentile" : 1.0,
> "Get_99th_percentile" : 1.0,
> {code}
> and
> {code}
>"Namespace_hbase_table_meta_region_1588230740_metric_get_num_ops" : 103,
> "Namespace_hbase_table_meta_region_1588230740_metric_get_min" : 450,
> "Namespace_hbase_table_meta_region_1588230740_metric_get_max" : 470,
> "Namespace_hbase_table_meta_region_1588230740_metric_get_mean" :
> 450.19417475728153,
> "Namespace_hbase_table_meta_region_1588230740_metric_get_median" : 460.0,
> "Namespace_hbase_table_meta_region_1588230740_metric_get_75th_percentile"
> : 470.0,
> "Namespace_hbase_table_meta_region_1588230740_metric_get_95th_percentile"
> : 470.0,
> "Namespace_hbase_table_meta_region_1588230740_metric_get_99th_percentile"
> : 470.0,
> {code}
> The problem is that the report values for the region server shows the
> latency, versus the reported values for the region shows the response sizes.
> There is no way of telling this without reading the source code.
> I think we should deprecate response size histograms in favor of latency
> histograms.
> See also HBASE-15376.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-15329) Cross-Site Scripting: Reflected in table.jsp
[
https://issues.apache.org/jira/browse/HBASE-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-15329.
---
Resolution: Fixed
Hadoop Flags: Reviewed
> Cross-Site Scripting: Reflected in table.jsp
>
>
> Key: HBASE-15329
> URL: https://issues.apache.org/jira/browse/HBASE-15329
> Project: HBase
> Issue Type: Bug
> Components: security
>Reporter: stack
>Assignee: Samir Ahmic
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15329_v0.patch
>
>
> Minor issue where we write back table name in a few places. Should clean it
> up:
> {code}
> } else {
> out.write("\nTable: ");
> out.print( fqtn );
> out.write("\n");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15384) Avoid using '/tmp' directory in TestBulkLoad
Heng Chen created HBASE-15384: - Summary: Avoid using '/tmp' directory in TestBulkLoad Key: HBASE-15384 URL: https://issues.apache.org/jira/browse/HBASE-15384 Project: HBase Issue Type: Sub-task Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15326) NPE in TestHRegion.testBatchPut_whileNoRowLocksHeld
Heng Chen created HBASE-15326:
-
Summary: NPE in TestHRegion.testBatchPut_whileNoRowLocksHeld
Key: HBASE-15326
URL: https://issues.apache.org/jira/browse/HBASE-15326
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
{code}
java.lang.NullPointerException
at
org.apache.hadoop.metrics2.lib.MutableHistogram.updateSnapshotMetrics(MutableHistogram.java:72)
at
org.apache.hadoop.metrics2.lib.MutableRangeHistogram.snapshot(MutableRangeHistogram.java:59)
at
org.apache.hadoop.metrics2.lib.DynamicMetricsRegistry.snapshot(DynamicMetricsRegistry.java:391)
at
org.apache.hadoop.hbase.metrics.BaseSourceImpl.getMetrics(BaseSourceImpl.java:146)
at
org.apache.hadoop.hbase.test.MetricsAssertHelperImpl.getMetrics(MetricsAssertHelperImpl.java:243)
at
org.apache.hadoop.hbase.test.MetricsAssertHelperImpl.getCounter(MetricsAssertHelperImpl.java:201)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testBatchPut_whileNoRowLocksHeld(TestHRegion.java:1498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
{code}
It seems to be introduced after HBASE-15222, [~eclark]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15308) Flakey TestSplitWalDataLoss on branch-1.1
Heng Chen created HBASE-15308: - Summary: Flakey TestSplitWalDataLoss on branch-1.1 Key: HBASE-15308 URL: https://issues.apache.org/jira/browse/HBASE-15308 Project: HBase Issue Type: Sub-task Reporter: Heng Chen It happens during HBASE-15169 QA test, see https://builds.apache.org/job/PreCommit-HBASE-Build/628/artifact/patchprocess/patch-unit-hbase-server-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HBASE-Build/547/artifact/patchprocess/patch-unit-hbase-server-jdk1.8.0_72.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15288) Flakey TestMasterMetrics.testClusterRequests on branch-1.1
Heng Chen created HBASE-15288: - Summary: Flakey TestMasterMetrics.testClusterRequests on branch-1.1 Key: HBASE-15288 URL: https://issues.apache.org/jira/browse/HBASE-15288 Project: HBase Issue Type: Sub-task Reporter: Heng Chen I found it during HBASE-15169, Let me see what's happening. https://builds.apache.org/job/PreCommit-HBASE-Build/586/testReport/org.apache.hadoop.hbase.master/TestMasterMetrics/testClusterRequests/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-15182) unify normalizer switch
[ https://issues.apache.org/jira/browse/HBASE-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen resolved HBASE-15182. --- Resolution: Invalid > unify normalizer switch > --- > > Key: HBASE-15182 > URL: https://issues.apache.org/jira/browse/HBASE-15182 > Project: HBase > Issue Type: Sub-task >Reporter: Heng Chen > Fix For: 2.0.0, 1.3.0 > > > After HBASE-15128, we will have an uniform way to do switch. Let's unify > normalizer into it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15280) Scan metrics should not ignore empty rows.
Heng Chen created HBASE-15280: - Summary: Scan metrics should not ignore empty rows. Key: HBASE-15280 URL: https://issues.apache.org/jira/browse/HBASE-15280 Project: HBase Issue Type: Bug Reporter: Heng Chen It is come from HBASE-15267. we found that as for empty result, GET increment the readRequestCount, but Scan does not do it. Detail information, please see HBASE-15267 comments. This jira is to fix it. We should record each request we do. At least, we could make Scan metrics behavior be consistent with Get. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-15178) metrics on web UI sometimes flakey
[ https://issues.apache.org/jira/browse/HBASE-15178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen resolved HBASE-15178. --- Resolution: Duplicate > metrics on web UI sometimes flakey > -- > > Key: HBASE-15178 > URL: https://issues.apache.org/jira/browse/HBASE-15178 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.6 >Reporter: Heng Chen > Attachments: 70DD704D-7E05-49BA-AC84-0646671F67AA.png > > > see attachement -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15182) unify normalizer switch
Heng Chen created HBASE-15182: - Summary: unify normalizer switch Key: HBASE-15182 URL: https://issues.apache.org/jira/browse/HBASE-15182 Project: HBase Issue Type: Sub-task Reporter: Heng Chen After HBASE-15128, we will have an uniform way to do switch. Let's unify normalizer into it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15178) metrics on web UI sometimes flakey
Heng Chen created HBASE-15178: - Summary: metrics on web UI sometimes flakey Key: HBASE-15178 URL: https://issues.apache.org/jira/browse/HBASE-15178 Project: HBase Issue Type: Bug Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15108) TestReplicationAdmin failed on branch-1.0
Heng Chen created HBASE-15108: - Summary: TestReplicationAdmin failed on branch-1.0 Key: HBASE-15108 URL: https://issues.apache.org/jira/browse/HBASE-15108 Project: HBase Issue Type: Bug Reporter: Heng Chen I notice it on HBASE-15095. See https://builds.apache.org/job/PreCommit-HBASE-Build/95/artifact/patchprocess/patch-unit-hbase-server-jdk1.8.0.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15049) AuthTypes.NONE cause exception after HS2 start
Heng Chen created HBASE-15049:
-
Summary: AuthTypes.NONE cause exception after HS2 start
Key: HBASE-15049
URL: https://issues.apache.org/jira/browse/HBASE-15049
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
I set {{hive.server2.authentication}} to be {{NONE}}
After HS2 start, i see exception is log below:
{code}
2015-12-29 16:58:42,339 ERROR [HiveServer2-Handler-Pool: Thread-31]:
server.TThreadPoolServer (TThreadPoolServer.java:run(296)) - Error occurred
during processing of message.
java.lang.RuntimeException:
org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in
the stream
at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no
sasl data in the stream
at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:328)
at
org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
... 4 more
{code}
IMO the problem is we use Sasl transport when authType is NONE,
{code:title=HiveAuthFactory.java}
public TTransportFactory getAuthTransFactory() throws LoginException {
TTransportFactory transportFactory;
if (authTypeStr.equalsIgnoreCase(AuthTypes.KERBEROS.getAuthName())) {
try {
transportFactory =
saslServer.createTransportFactory(getSaslProperties());
} catch (TTransportException e) {
throw new LoginException(e.getMessage());
}
} else if (authTypeStr.equalsIgnoreCase(AuthTypes.NONE.getAuthName())) {
transportFactory = PlainSaslHelper.getPlainTransportFactory(authTypeStr);
} else if (authTypeStr.equalsIgnoreCase(AuthTypes.LDAP.getAuthName())) {
transportFactory = PlainSaslHelper.getPlainTransportFactory(authTypeStr);
} else if (authTypeStr.equalsIgnoreCase(AuthTypes.PAM.getAuthName())) {
transportFactory = PlainSaslHelper.getPlainTransportFactory(authTypeStr);
} else if (authTypeStr.equalsIgnoreCase(AuthTypes.NOSASL.getAuthName())) {
transportFactory = new TTransportFactory();
} else if (authTypeStr.equalsIgnoreCase(AuthTypes.CUSTOM.getAuthName())) {
transportFactory = PlainSaslHelper.getPlainTransportFactory(authTypeStr);
} else {
throw new LoginException("Unsupported authentication type " +
authTypeStr);
}
return transportFactory;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-15049) AuthTypes.NONE cause exception after HS2 start
[
https://issues.apache.org/jira/browse/HBASE-15049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-15049.
---
Resolution: Invalid
> AuthTypes.NONE cause exception after HS2 start
> --
>
> Key: HBASE-15049
> URL: https://issues.apache.org/jira/browse/HBASE-15049
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>
> I set {{hive.server2.authentication}} to be {{NONE}}
> After HS2 start, i see exception in log below:
> {code}
> 2015-12-29 16:58:42,339 ERROR [HiveServer2-Handler-Pool: Thread-31]:
> server.TThreadPoolServer (TThreadPoolServer.java:run(296)) - Error occurred
> during processing of message.
> java.lang.RuntimeException:
> org.apache.thrift.transport.TSaslTransportException: No data or no sasl data
> in the stream
> at
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no
> sasl data in the stream
> at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:328)
> at
> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
> at
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
> ... 4 more
> {code}
> IMO the problem is we use Sasl transport when authType is NONE,
> {code:title=HiveAuthFactory.java}
> public TTransportFactory getAuthTransFactory() throws LoginException {
> TTransportFactory transportFactory;
> if (authTypeStr.equalsIgnoreCase(AuthTypes.KERBEROS.getAuthName())) {
> try {
> transportFactory =
> saslServer.createTransportFactory(getSaslProperties());
> } catch (TTransportException e) {
> throw new LoginException(e.getMessage());
> }
> } else if (authTypeStr.equalsIgnoreCase(AuthTypes.NONE.getAuthName())) {
> transportFactory =
> PlainSaslHelper.getPlainTransportFactory(authTypeStr);
> } else if (authTypeStr.equalsIgnoreCase(AuthTypes.LDAP.getAuthName())) {
> transportFactory =
> PlainSaslHelper.getPlainTransportFactory(authTypeStr);
> } else if (authTypeStr.equalsIgnoreCase(AuthTypes.PAM.getAuthName())) {
> transportFactory =
> PlainSaslHelper.getPlainTransportFactory(authTypeStr);
> } else if (authTypeStr.equalsIgnoreCase(AuthTypes.NOSASL.getAuthName())) {
> transportFactory = new TTransportFactory();
> } else if (authTypeStr.equalsIgnoreCase(AuthTypes.CUSTOM.getAuthName())) {
> transportFactory =
> PlainSaslHelper.getPlainTransportFactory(authTypeStr);
> } else {
> throw new LoginException("Unsupported authentication type " +
> authTypeStr);
> }
> return transportFactory;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen reopened HBASE-14684: --- revert branch-1. It seems some problems about some tests > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: 14684.branch-1.txt, 14684.branch-1.txt, > 14684.branch-1.txt, HBASE-14684-branch-1.2.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1.patch, HBASE-14684-branch-1.patch, > HBASE-14684-branch-1_v1.patch, HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14949) Skip duplicate entries when replay WAL.
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen resolved HBASE-14949. --- Resolution: Invalid > Skip duplicate entries when replay WAL. > --- > > Key: HBASE-14949 > URL: https://issues.apache.org/jira/browse/HBASE-14949 > Project: HBase > Issue Type: Sub-task >Reporter: Heng Chen > Attachments: HBASE-14949.patch > > > As HBASE-14004 design, there will be duplicate entries in different WAL. It > happens when one hflush failed, we will close old WAL with 'acked hflushed' > length, then open a new WAL and write the unacked hlushed entries into it. > So there maybe some overlap between old WAL and new WAL. > We should skip the duplicate entries when replay. I think it has no harm to > current logic, maybe we do it first. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-14949) Skip duplicate entries when replay WAL.
[ https://issues.apache.org/jira/browse/HBASE-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen reopened HBASE-14949: --- > Skip duplicate entries when replay WAL. > --- > > Key: HBASE-14949 > URL: https://issues.apache.org/jira/browse/HBASE-14949 > Project: HBase > Issue Type: Sub-task >Reporter: Heng Chen > Attachments: HBASE-14949.patch > > > As HBASE-14004 design, there will be duplicate entries in different WAL. It > happens when one hflush failed, we will close old WAL with 'acked hflushed' > length, then open a new WAL and write the unacked hlushed entries into it. > So there maybe some overlap between old WAL and new WAL. > We should skip the duplicate entries when replay. I think it has no harm to > current logic, maybe we do it first. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14949) Skip duplicate entries when replay WAL.
Heng Chen created HBASE-14949: - Summary: Skip duplicate entries when replay WAL. Key: HBASE-14949 URL: https://issues.apache.org/jira/browse/HBASE-14949 Project: HBase Issue Type: Improvement Reporter: Heng Chen As HBASE-14004 design, there will be duplicate entries in different WAL. It happens when one hflush failed, we will close old WAL with 'acked hflushed' length, then open a new WAL and write the unacked hlushed entries into it. So there maybe some overlap between old WAL and new WAL. We should skip the duplicate entries when replay. I think it has no harm to current logic, maybe we do it first. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14897) TestTableLockManager.testReapAllTableLocks is flakey
Heng Chen created HBASE-14897:
-
Summary: TestTableLockManager.testReapAllTableLocks is flakey
Key: HBASE-14897
URL: https://issues.apache.org/jira/browse/HBASE-14897
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
It comes from email list which [~stack] post.
This is the some relates QA information.
https://builds.apache.org/view/H-L/view/HBase/job/HBase-Trunk_matrix/512/jdk=latest1.8,label=Hadoop/testReport/org.apache.hadoop.hbase.master/TestTableLockManager/testReapAllTableLocks/
The reason is here.
{code}
writeLocksObtained.await();
writeLocksAttempted.await();
{code}
writeLocksAttempted maybe count down to 0 before created node on ZK, and main
thread will go on to run lockManager.reapWriteLocks(), And after that node was
created on ZK, so relates lock acquire will timeout.
I upload a patch which can reproduce this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14843) TestWALProcedureStore.testLoad is flakey
Heng Chen created HBASE-14843: - Summary: TestWALProcedureStore.testLoad is flakey Key: HBASE-14843 URL: https://issues.apache.org/jira/browse/HBASE-14843 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Heng Chen I see it twice recently, see. https://builds.apache.org/job/PreCommit-HBASE-Build/16589//testReport/org.apache.hadoop.hbase.procedure2.store.wal/TestWALProcedureStore/testLoad/ https://builds.apache.org/job/PreCommit-HBASE-Build/16532/testReport/org.apache.hadoop.hbase.procedure2.store.wal/TestWALProcedureStore/testLoad/ Let's see what's happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14815) TestMobExportSnapshot.testExportFailure timeout occasionally
Heng Chen created HBASE-14815: - Summary: TestMobExportSnapshot.testExportFailure timeout occasionally Key: HBASE-14815 URL: https://issues.apache.org/jira/browse/HBASE-14815 Project: HBase Issue Type: Bug Reporter: Heng Chen On master, TestMobExportSnapshot.testExportFailure timeout occasionally. See https://builds.apache.org/job/PreCommit-HBASE-Build/16514//testReport/org.apache.hadoop.hbase.snapshot/TestMobExportSnapshot/testExportFailure/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14659) Fix flakey TestHFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-14659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen resolved HBASE-14659. --- Resolution: Won't Fix > Fix flakey TestHFileOutputFormat > > > Key: HBASE-14659 > URL: https://issues.apache.org/jira/browse/HBASE-14659 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen >Assignee: Heng Chen > > As i said in HBASE-14654 > This testcase was hanged twice recently. > the two QA information: > https://builds.apache.org/job/PreCommit-HBASE-Build/16118/console > https://builds.apache.org/job/PreCommit-HBASE-Build/16117/console > I notice that the two QA running simultaneously on same machine H0. > I doubt If there are some common resources the two QA conflicts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14265) we should forbid creating table using 'hbase' namespace except by superuser
[
https://issues.apache.org/jira/browse/HBASE-14265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen resolved HBASE-14265.
---
Resolution: Invalid
> we should forbid creating table using 'hbase' namespace except by superuser
> ---
>
> Key: HBASE-14265
> URL: https://issues.apache.org/jira/browse/HBASE-14265
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>Assignee: Heng Chen
> Attachments: HBASE-14265.patch, HBASE-14265_v2.patch,
> HBASE-14265_v3.patch, HBASE-14265_v4.patch
>
>
> Now, there is no limit for users who can create table under 'hbase'
> NameSpace. I think it has some risk.
> Because we use {{TableName.systemTable}} to decide whether this table is
> System or not.
> But as code, {{TableName.systemTable}} will be true, if NS equals "hbase'
> {code}
> if (Bytes.equals(NamespaceDescriptor.SYSTEM_NAMESPACE_NAME, namespace)) {
> this.namespace = NamespaceDescriptor.SYSTEM_NAMESPACE_NAME;
> this.namespaceAsString =
> NamespaceDescriptor.SYSTEM_NAMESPACE_NAME_STR;
> this.systemTable = true;
> }
> {code}
>
> And we treat system table and normal table differently.
> For example, https://issues.apache.org/jira/browse/HBASE-14257 will flush
> fast if table belong to system table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14703) update the per-region stats twice for the call on return
Heng Chen created HBASE-14703:
-
Summary: update the per-region stats twice for the call on return
Key: HBASE-14703
URL: https://issues.apache.org/jira/browse/HBASE-14703
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
In {{AsyncProcess.SingleServerRequestRunnable}}, it seems we update
serverStatistics twice.
The first one is that we wrapper {{RetryingCallable}} by
{{StatsTrackingRpcRetryingCaller}}, and do serverStatistics update when we call
{{callWithRetries}} and {{callWithoutRetries}}. Relates code like below:
{code}
@Override
public T callWithRetries(RetryingCallable callable, int callTimeout)
throws IOException, RuntimeException {
T result = delegate.callWithRetries(callable, callTimeout);
return updateStatsAndUnwrap(result, callable);
}
@Override
public T callWithoutRetries(RetryingCallable callable, int callTimeout)
throws IOException, RuntimeException {
T result = delegate.callWithRetries(callable, callTimeout);
return updateStatsAndUnwrap(result, callable);
}
{code}
The secondary one is after we get response, in {{receiveMultiAction}}, we do
update again.
{code}
// update the stats about the region, if its a user table. We don't want to
slow down
// updates to meta tables, especially from internal updates (master, etc).
if (AsyncProcess.this.connection.getStatisticsTracker() != null) {
result = ResultStatsUtil.updateStats(result,
AsyncProcess.this.connection.getStatisticsTracker(), server, regionName);
}
{code}
It seems that {{StatsTrackingRpcRetryingCaller}} is NOT necessary, remove it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
Heng Chen created HBASE-14684: - Summary: Try to remove all MiniMapReduceCluster in unit tests Key: HBASE-14684 URL: https://issues.apache.org/jira/browse/HBASE-14684 Project: HBase Issue Type: Improvement Reporter: Heng Chen As discussion in dev list, we will try to do MR job without MiniMapReduceCluster. Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14662) Fix NPE in HFileOutputFormat2
Heng Chen created HBASE-14662:
-
Summary: Fix NPE in HFileOutputFormat2
Key: HBASE-14662
URL: https://issues.apache.org/jira/browse/HBASE-14662
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
When i dig in HBASE-14659, i run testWritingPEData. There are a lot of NPE
thrown and testcase run a long time.
The reason is that, in {{HFileOutputFormat2}}
{code}
HRegionLocation loc = null;
String tableName = conf.get(OUTPUT_TABLE_NAME_CONF_KEY);
try (Connection connection =
ConnectionFactory.createConnection(conf);
RegionLocator locator =
connection.getRegionLocator(TableName.valueOf(tableName)))
{
loc = locator.getRegionLocation(rowKey);
} catch (Throwable e) {
LOG.warn("there's something wrong when locating rowkey: " +
Bytes.toString(rowKey), e);
loc = null;
}
{code}
Because we did not set {{OUTPUT_TABLE_NAME_CONF_KEY}}, So tableName is null,
So NPE thrown.
And RegionLocator will create connection which RegionLocator use to find region
location. Because zk is not start in this testcase, So it will retry many
times.
But all this actions is nesseray, we can skip create connection by check
whether tableName is null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14654) Reenable TestMultiParallel#testActiveThreadsCount
Heng Chen created HBASE-14654: - Summary: Reenable TestMultiParallel#testActiveThreadsCount Key: HBASE-14654 URL: https://issues.apache.org/jira/browse/HBASE-14654 Project: HBase Issue Type: Bug Reporter: Heng Chen It was disabled in HBASE-14642, this issue should reenable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14659) Fix flakey TestHFileOutputFormat
Heng Chen created HBASE-14659: - Summary: Fix flakey TestHFileOutputFormat Key: HBASE-14659 URL: https://issues.apache.org/jira/browse/HBASE-14659 Project: HBase Issue Type: Bug Reporter: Heng Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14455) Try to get rid of unused HConnection instance
Heng Chen created HBASE-14455:
-
Summary: Try to get rid of unused HConnection instance
Key: HBASE-14455
URL: https://issues.apache.org/jira/browse/HBASE-14455
Project: HBase
Issue Type: Improvement
Reporter: Heng Chen
Priority: Minor
After HBASE-14361, we get rid of HConnection Instance in ReplicationSink in
standalone mode. But there are still three HConnection Instance, i will try to
remove unused ones.
The three instance is created
{code}
6 2015-09-21 16:05:36,401 INFO [10.0.3.80:60429.activeMasterManager]
client.ConnectionImplementation:.
7 java.lang.Throwable
8 >---at
org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:217)
9 >---at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
10 >---at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
11 >---at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
12 >---at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
13 >---at
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:231)
14 >---at
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:118)
15 >---at
org.apache.hadoop.hbase.master.ServerManager.(ServerManager.java:222)
16 >---at
org.apache.hadoop.hbase.master.ServerManager.(ServerManager.java:212)
17 >---at
org.apache.hadoop.hbase.master.HMaster.createServerManager(HMaster.java:853)
18 >---at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:661)
19 >---at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:180)
20 >---at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1670)
21 >---at java.lang.Thread.run(Thread.java:745)
22 2015-09-21 16:05:36,401 INFO [M:0;10.0.3.80:60429]
client.ConnectionImplementation:.
23 java.lang.Throwable
24 >---at
org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:217)
25 >---at
org.apache.hadoop.hbase.client.ConnectionUtils$1.(ConnectionUtils.java:128)
26 >---at
org.apache.hadoop.hbase.client.ConnectionUtils.createShortCircuitConnection(ConnectionUtils.java:128)
27 >---at
org.apache.hadoop.hbase.regionserver.HRegionServer.createClusterConnection(HRegionServer.java:686)
28 >---at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupClusterConnection(HRegionServer.java:717)
29 >---at
org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:730)
30 >---at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:885)
31 >---at
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.run(HMasterCommandLine.java:311)
32 >---at java.lang.Thread.run(Thread.java:745)
33 2015-09-21 16:05:36,401 INFO [RS:0;10.0.3.80:60431]
client.ConnectionImplementation:.
34 java.lang.Throwable
35 >---at
org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:217)
36 >---at
org.apache.hadoop.hbase.client.ConnectionUtils$1.(ConnectionUtils.java:128)
37 >---at
org.apache.hadoop.hbase.client.ConnectionUtils.createShortCircuitConnection(ConnectionUtils.java:128)
38 >---at
org.apache.hadoop.hbase.regionserver.HRegionServer.createClusterConnection(HRegionServer.java:686)
39 >---at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupClusterConnection(HRegionServer.java:717)
40 >---at
org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:730)
41 >---at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:885)
42 >---at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14265) we should forbidden create table using 'hbase' namespace
Heng Chen created HBASE-14265:
-
Summary: we should forbidden create table using 'hbase' namespace
Key: HBASE-14265
URL: https://issues.apache.org/jira/browse/HBASE-14265
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
Now, there is no limit for users who can create table under 'hbase' NameSpace.
I think it has some risk.
Because we use {{TableName.systemTable}} to decide whether this table is System
or not.
But as code, {{TableName.systemTable}} will be true, if NS equals hbase'
{code}
if (Bytes.equals(NamespaceDescriptor.SYSTEM_NAMESPACE_NAME, namespace)) {
this.namespace = NamespaceDescriptor.SYSTEM_NAMESPACE_NAME;
this.namespaceAsString = NamespaceDescriptor.SYSTEM_NAMESPACE_NAME_STR;
this.systemTable = true;
}
{code}
And we treat system table and normal table differently.
For example, https://issues.apache.org/jira/browse/HBASE-14257 will flush fast
if table belong to system table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14230) replace reflection in FSHlog with HdfsDataOutputStream#getCurrentBlockReplication()
Heng Chen created HBASE-14230:
-
Summary: replace reflection in FSHlog with
HdfsDataOutputStream#getCurrentBlockReplication()
Key: HBASE-14230
URL: https://issues.apache.org/jira/browse/HBASE-14230
Project: HBase
Issue Type: Improvement
Components: wal
Reporter: Heng Chen
Priority: Minor
As comment TODO said, we use
{{HdfsDataOutputStream#getCurrentBlockReplication}} and
{{DFSOutputStream.getPipeLine}} to replace reflection in FSHlog
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14203) remove duplicate code getTableDescriptor in HTable
Heng Chen created HBASE-14203:
-
Summary: remove duplicate code getTableDescriptor in HTable
Key: HBASE-14203
URL: https://issues.apache.org/jira/browse/HBASE-14203
Project: HBase
Issue Type: Improvement
Reporter: Heng Chen
Priority: Trivial
As TODO in comment said,
{{HTable.getTableDescriptor}} is same as {{HAdmin.getTableDescriptor}}. remove
the duplicate code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14189) BlockCache options should consider CF Level BlockCacheEnabled setting
Heng Chen created HBASE-14189:
-
Summary: BlockCache options should consider CF Level
BlockCacheEnabled setting
Key: HBASE-14189
URL: https://issues.apache.org/jira/browse/HBASE-14189
Project: HBase
Issue Type: Improvement
Components: BlockCache
Reporter: Heng Chen
While using BlockCache, we use {{cacheDataOnRead}}({{cacheDataOnWrite}})
represents for whether we should cache block after read(write) block from(to)
hdfs. We should honour BC setting and CF Level cache setting while using
BlockCache.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14178) regionserver blocks because of waiting for offsetLock
Heng Chen created HBASE-14178:
-
Summary: regionserver blocks because of waiting for offsetLock
Key: HBASE-14178
URL: https://issues.apache.org/jira/browse/HBASE-14178
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
Priority: Critical
My regionserver blocks, and all client rpc timeout.
I print the regionserver's jstack, it seems a lot of thread was blocked for
waiting offsetLock, detail infomation belows:
{code}
B.DefaultRpcServer.handler=2,queue=2,port=60020 #82 daemon prio=5 os_prio=0
tid=0x01827000 nid=0x2cdc in Object.wait() [0x7f3831b72000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.hadoop.hbase.util.IdLock.getLockEntry(IdLock.java:79)
- locked 0x000773af7c18 (a
org.apache.hadoop.hbase.util.IdLock$Entry)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:352)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:253)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:524)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:572)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
at
org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:269)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:695)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:683)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:533)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:140)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3889)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3969)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3847)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3820)
- locked 0x0005e5c55ad0 (a
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3807)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4779)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4753)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2916)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29583)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- 0x0005e5c55c08 (a
java.util.concurrent.locks.ReentrantLock$NonfairSync)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14113) Compile failed on master
Heng Chen created HBASE-14113:
-
Summary: Compile failed on master
Key: HBASE-14113
URL: https://issues.apache.org/jira/browse/HBASE-14113
Project: HBase
Issue Type: Bug
Components: build
Reporter: Heng Chen
mvn package -DskipTests
Compile failed!
{code:bash}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on
project hbase-client: Compilation failure: Compilation failure:
[ERROR]
/Users/chenheng/apache/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/BinaryComparator.java:[54,3]
method does not override or implement a method from a supertype
[ERROR]
/Users/chenheng/apache/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/NullComparator.java:[64,3]
method does not override or implement a method from a supertype
[ERROR]
/Users/chenheng/apache/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/LongComparator.java:[51,5]
method does not override or implement a method from a supertype
[ERROR]
/Users/chenheng/apache/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/BitComparator.java:[137,3]
method does not override or implement a method from a supertype
[ERROR]
/Users/chenheng/apache/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/BinaryPrefixComparator.java:[56,3]
method does not override or implement a method from a supertype
{code}
My java version:
{code}
java version 1.8.0_40
Java(TM) SE Runtime Environment (build 1.8.0_40-b25)
Java HotSpot(TM) 64-Bit Server VM (build 25.40-b25, mixed mode)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
