[jira] [Commented] (HBASE-14254) Wrong error message when throwing NamespaceNotFoundException in shell
[
https://issues.apache.org/jira/browse/HBASE-14254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702863#comment-14702863
]
Liu Shaohui commented on HBASE-14254:
-
[~chenheng]
This problem is introduced in HBASE-14119
Wrong error message when throwing NamespaceNotFoundException in shell
-
Key: HBASE-14254
URL: https://issues.apache.org/jira/browse/HBASE-14254
Project: HBase
Issue Type: Improvement
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Minor
Wrong error message when throwing NamespaceNotFoundException in shell
{code}
hbase(main):004:0 create 'ns:t1', {NAME = 'f1'}
ERROR: Unknown namespace ns:t1!
{code}
The namespace shoud be {color:red}ns {color}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13376) Improvements to Stochastic load balancer
[ https://issues.apache.org/jira/browse/HBASE-13376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702998#comment-14702998 ] Hudson commented on HBASE-13376: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1041 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1041/]) HBASE-13376 Improvements to Stochastic load balancer (Vandana Ayyalasomayajula) (tedyu: rev 2fd9c8df3b261dd596f68fff1c09c803da1ccdc3) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/RegionLocationFinder.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java Improvements to Stochastic load balancer Key: HBASE-13376 URL: https://issues.apache.org/jira/browse/HBASE-13376 Project: HBase Issue Type: Improvement Components: Balancer Affects Versions: 1.0.0, 0.98.12 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Fix For: 2.0.0, 0.98.14, 1.3.0 Attachments: 13376-v2.txt, 13376-v5.patch, 13376_4.patch, HBASE-13376.patch, HBASE-13376_0.98.txt, HBASE-13376_0.98_v2.patch, HBASE-13376_0.txt, HBASE-13376_1.txt, HBASE-13376_1_1.txt, HBASE-13376_2.patch, HBASE-13376_2_branch-1.patch, HBASE-13376_3.patch, HBASE-13376_3.patch, HBASE-13376_4.patch, HBASE-13376_5_branch-1.patch, HBASE-13376_6_branch-1.patch, HBASE-13376_98.patch, HBASE-13376_branch-1.patch, HBASE-13376_v3_0.98.patch, HBASE-13376_v4_0.98.patch There are two things this jira tries to address: 1. The locality picker in the stochastic balancer does not pick regions with least locality as candidates for swap/move. So when any user configures locality cost in the configs, the balancer does not always seems to move regions with bad locality. 2. When a cluster has equal number of loaded regions, it always picks the first one. It should pick a random region on one of the equally loaded servers. This improves a chance of finding a good candidate, when load picker is invoked several times. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14230) replace reflection in FSHlog with HdfsDataOutputStream#getCurrentBlockReplication()
[
https://issues.apache.org/jira/browse/HBASE-14230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703042#comment-14703042
]
Hadoop QA commented on HBASE-14230:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12751249/HBASE-14230.patch
against master branch at commit 1bb9e3ae966853db8c4138b4abbe14636d7592db.
ATTACHMENT ID: 12751249
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestAcidGuarantees
org.apache.hadoop.hbase.regionserver.TestCompactionState
org.apache.hadoop.hbase.regionserver.TestSeekOptimizations
org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.security.access.TestCellACLWithMultipleVersions
org.apache.hadoop.hbase.security.access.TestNamespaceCommands
org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss
org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.regionserver.TestRegionReplicas
org.apache.hadoop.hbase.TestFullLogReconstruction
org.apache.hadoop.hbase.regionserver.TestRegionServerReportForDuty
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster
org.apache.hadoop.hbase.filter.TestFuzzyRowAndColumnRangeFilter
org.apache.hadoop.hbase.security.access.TestAccessController2
org.apache.hadoop.hbase.regionserver.TestDeleteMobTable
org.apache.hadoop.hbase.security.access.TestAccessController
org.apache.hadoop.hbase.security.access.TestAccessControlFilter
org.apache.hadoop.hbase.regionserver.TestRegionFavoredNodes
org.apache.hadoop.hbase.security.access.TestTablePermissions
org.apache.hadoop.hbase.regionserver.TestSCVFWithMiniCluster
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove
org.apache.hadoop.hbase.regionserver.TestScannerWithBulkload
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15166//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15166//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15166//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15166//console
This message is automatically generated.
replace reflection in FSHlog with
HdfsDataOutputStream#getCurrentBlockReplication()
---
Key: HBASE-14230
URL: https://issues.apache.org/jira/browse/HBASE-14230
Project: HBase
Issue Type: Improvement
Components: wal
Reporter: Heng Chen
Priority: Minor
Attachments: HBASE-14230.patch
As comment TODO said, we use
{{HdfsDataOutputStream#getCurrentBlockReplication}} and
{{DFSOutputStream.getPipeLine}} to replace
[jira] [Updated] (HBASE-14254) Wrong error message when throwing NamespaceNotFoundException in shell
[
https://issues.apache.org/jira/browse/HBASE-14254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Shaohui updated HBASE-14254:
Attachment: HBASE-14254-v001.diff
Fix it in commands.rb
Wrong error message when throwing NamespaceNotFoundException in shell
-
Key: HBASE-14254
URL: https://issues.apache.org/jira/browse/HBASE-14254
Project: HBase
Issue Type: Improvement
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Minor
Attachments: HBASE-14254-v001.diff
Wrong error message when throwing NamespaceNotFoundException in shell
{code}
hbase(main):004:0 create 'ns:t1', {NAME = 'f1'}
ERROR: Unknown namespace ns:t1!
{code}
The namespace shoud be {color:red}ns {color}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14241) Fix deadlock during cluster shutdown due to concurrent connection close
[ https://issues.apache.org/jira/browse/HBASE-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14241: --- Summary: Fix deadlock during cluster shutdown due to concurrent connection close (was: Deadlock during HBaseTestingUtility#shutdownMiniHBaseCluster) Fix deadlock during cluster shutdown due to concurrent connection close --- Key: HBASE-14241 URL: https://issues.apache.org/jira/browse/HBASE-14241 Project: HBase Issue Type: Bug Affects Versions: 1.0.2 Reporter: Andrew Purtell Assignee: Ted Yu Priority: Critical Fix For: 2.0.0, 1.0.2, 1.2.0, 1.3.0, 1.1.3 Attachments: 14241-v2.txt, 14241-v3.txt, 14241-v4.txt, 14241-v5.txt, deadlock.txt.gz Caught while testing branch-1.0, shutting down TestMasterMetricsWrapper. Found one Java-level deadlock: = MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0: waiting to lock monitor 0x7f2a040051c8 (object 0x0007e36108a8, a org.apache.hadoop.hbase.util.PoolMap), which is held by M:0;ip-10-32-130-237:55342 M:0;ip-10-32-130-237:55342: waiting to lock monitor 0x7f2a04005118 (object 0x0007e3610b00, a org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection), which is held by MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0 Full stack dump and deadlock debug output attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[
https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702954#comment-14702954
]
Hadoop QA commented on HBASE-13158:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12751222/HBASE-13158_V6.patch
against master branch at commit 1bb9e3ae966853db8c4138b4abbe14636d7592db.
ATTACHMENT ID: 12751222
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.zookeeper.TestZooKeeperNodeTracker
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.master.TestRollingRestart
org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat
{color:red}-1 core zombie tests{color}. There are 9 zombie test(s):
at
org.apache.hadoop.hbase.TestRegionRebalancing.testRebalanceOnRegionServerNumberChange(TestRegionRebalancing.java:152)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15164//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15164//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15164//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15164//console
This message is automatically generated.
When client supports CellBlock, return the result Cells as controller payload
for get(Get) API also
---
Key: HBASE-13158
URL: https://issues.apache.org/jira/browse/HBASE-13158
Project: HBase
Issue Type: Improvement
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 2.0.0, 1.3.0
Attachments: 13158v4.suggestion.txt, HBASE-13158.patch,
HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch,
HBASE-13158_V6.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14215) Default cost used for PrimaryRegionCountSkewCostFunction is not sufficient
[
https://issues.apache.org/jira/browse/HBASE-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Biju Nair updated HBASE-14215:
--
Description:
Current multiplier of 500 used in the stochastic balancer cost function
{{PrimaryRegionCountSkewCostFunction}} to calculate the cost of total primary
replication skew doesn't seem to be sufficient to prevent the skews (Refer
HBASE-14110). We would want the default cost to be a higher value so that skews
in primary region replica has higher cost. The following is the test result by
setting the multiplier value to 1 (same as the region replica rack cost
multiplier) on a 3 Rack 9 RS node cluster which seems to get the balancer
distribute the primaries uniformly.
*Initial Primary replica distribution - using the current multiplier*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9 124
r3n10 135
r3n11 124
r3n9129
*After long duration of read writes - using current multiplier*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9124
r3n10 135
r3n11 124
r3n9129
*After manual balancing*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9124
r3n10 135
r3n11 124
r3n9129
*Increased multiplier for primaryRegionCountSkewCost to 1*
r1n10 114
r1n11 113
r1n9114
r2n10 114
r2n11 114
r2n9113
r3n10 115
r3n11 115
r3n9115
Setting the {{PrimaryRegionCountSkewCostFunction}} multiplier value to 1
should help HBase general use.
was:
Current multiplier of 500 used in the stochastic balancer cost function
``PrimaryRegionCountSkewCostFunction`` to calculate the cost of total primary
replication skew doesn't seem to be sufficient to prevent the skews (Refer
HBASE-14110). We would want the default cost to be a higher value so that skews
in primary region replica has higher cost. The following is the test result by
setting the multiplier value to 1 (same as the region replica rack cost
multiplier) on a 3 Rack 9 RS node cluster which seems to get the balancer
distribute the primaries uniformly.
*Initial Primary replica distribution - using the current multiplier*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9 124
r3n10 135
r3n11 124
r3n9129
*After long duration of read writes - using current multiplier*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9124
r3n10 135
r3n11 124
r3n9129
*After manual balancing*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9124
r3n10 135
r3n11 124
r3n9129
*Increased multiplier for primaryRegionCountSkewCost to 1*
r1n10 114
r1n11 113
r1n9114
r2n10 114
r2n11 114
r2n9113
r3n10 115
r3n11 115
r3n9115
Setting the `PrimaryRegionCountSkewCostFunction` multiplier value to 1
should help HBase general use.
Default cost used for PrimaryRegionCountSkewCostFunction is not sufficient
---
Key: HBASE-14215
URL: https://issues.apache.org/jira/browse/HBASE-14215
Project: HBase
Issue Type: Bug
Components: Balancer
Reporter: Biju Nair
Priority: Minor
Attachments: 14215-v1.txt
Current multiplier of 500 used in the stochastic balancer cost function
{{PrimaryRegionCountSkewCostFunction}} to calculate the cost of total
primary replication skew doesn't seem to be sufficient to prevent the skews
(Refer HBASE-14110). We would want the default cost to be a higher value so
that skews in primary region replica has higher cost. The following is the
test result by setting the multiplier value to 1 (same as the region
replica rack cost multiplier) on a 3 Rack 9 RS node cluster which seems to
get the balancer distribute the primaries uniformly.
*Initial Primary replica distribution - using the current multiplier*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9 124
r3n10 135
r3n11 124
r3n9129
*After long duration of read writes - using current multiplier*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9124
r3n10 135
r3n11 124
r3n9129
*After manual balancing*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9124
r3n10 135
r3n11 124
r3n9129
*Increased multiplier for primaryRegionCountSkewCost to 1*
r1n10 114
r1n11 113
r1n9114
r2n10 114
r2n11 114
r2n9113
r3n10 115
r3n11 115
r3n9115
Setting the {{PrimaryRegionCountSkewCostFunction}} multiplier value to 1
should help HBase general use.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14241) Fix deadlock during cluster shutdown due to concurrent connection close
[
https://issues.apache.org/jira/browse/HBASE-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14241:
---
Resolution: Fixed
Fix Version/s: (was: 1.1.3)
1.1.2
Status: Resolved (was: Patch Available)
Thanks for the review, Andrew.
Fix deadlock during cluster shutdown due to concurrent connection close
---
Key: HBASE-14241
URL: https://issues.apache.org/jira/browse/HBASE-14241
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.2
Reporter: Andrew Purtell
Assignee: Ted Yu
Priority: Critical
Fix For: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.2
Attachments: 14241-v2.txt, 14241-v3.txt, 14241-v4.txt, 14241-v5.txt,
deadlock.txt.gz
Caught while testing branch-1.0, shutting down TestMasterMetricsWrapper.
Found one Java-level deadlock:
=
MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0:
waiting to lock monitor 0x7f2a040051c8 (object 0x0007e36108a8, a
org.apache.hadoop.hbase.util.PoolMap),
which is held by M:0;ip-10-32-130-237:55342
M:0;ip-10-32-130-237:55342:
waiting to lock monitor 0x7f2a04005118 (object 0x0007e3610b00, a
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),
which is held by MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0
Full stack dump and deadlock debug output attached.
Root cause:
In RpcClientImpl#close(), we obtain lock on connections first:
{code}
synchronized (connections) {
for (Connection conn : connections.values()) {
{code}
Then markClosed() tries to obtain lock on connection object:
{code}
if (!conn.isAlive()) {
conn.markClosed(new InterruptedIOException(RpcClient is closing));
conn.close();
{code}
Another thread, MetaServerShutdownHandler, calls
RpcClientImpl$Connection#setupIOstreams() where :
{code}
markClosed(e);
close();
{code}
Lock on connection object is obtained first, then lock on connections is
attempted, leading to deadlock:
{code}
synchronized (connections) {
connections.removeValue(remoteId, this);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12751) Allow RowLock to be reader writer
[ https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12751: -- Attachment: HBASE-12751-v20.patch The 'Anti-Pattern' fail is incorrect for this patch; this patch actually fixes an instance of this issue. The grep must be turning up this. Less fails. Let me rerun to see if we've settled on a small set of fails... will then dig in. Allow RowLock to be reader writer - Key: HBASE-12751 URL: https://issues.apache.org/jira/browse/HBASE-12751 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.0.0, 1.3.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 1.3.0 Attachments: HBASE-12751-v1.patch, HBASE-12751-v10.patch, HBASE-12751-v10.patch, HBASE-12751-v11.patch, HBASE-12751-v12.patch, HBASE-12751-v13.patch, HBASE-12751-v14.patch, HBASE-12751-v15.patch, HBASE-12751-v16.patch, HBASE-12751-v17.patch, HBASE-12751-v18.patch, HBASE-12751-v19 (1).patch, HBASE-12751-v19.patch, HBASE-12751-v2.patch, HBASE-12751-v20.patch, HBASE-12751-v20.patch, HBASE-12751-v3.patch, HBASE-12751-v4.patch, HBASE-12751-v5.patch, HBASE-12751-v6.patch, HBASE-12751-v7.patch, HBASE-12751-v8.patch, HBASE-12751-v9.patch, HBASE-12751.patch Right now every write operation grabs a row lock. This is to prevent values from changing during a read modify write operation (increment or check and put). However it limits parallelism in several different scenarios. If there are several puts to the same row but different columns or stores then this is very limiting. If there are puts to the same column then mvcc number should ensure a consistent ordering. So locking is not needed. However locking for check and put or increment is still needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12890) Provide a way to throttle the number of regions moved by the balancer
[ https://issues.apache.org/jira/browse/HBASE-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703188#comment-14703188 ] churro morales commented on HBASE-12890: Agreed, it does work for the stochastic load balancer. Reason why i suggested the patch was this will work for any balancer you decide to go with. The patch however has the downfall where the respective balancer creates plans for everything then we filter away..but we use the table level and not stochastic. Provide a way to throttle the number of regions moved by the balancer - Key: HBASE-12890 URL: https://issues.apache.org/jira/browse/HBASE-12890 Project: HBase Issue Type: Improvement Affects Versions: 0.98.10 Reporter: churro morales Assignee: churro morales Fix For: 2.0.0, 1.3.0, 0.98.15 Attachments: HBASE-12890.patch We have a very large cluster and we frequently add remove quite a few regionservers from our cluster. Whenever we do this the balancer moves thousands of regions at once. Instead we provide a configuration parameter: hbase.balancer.max.regions. This limits the number of regions that are balanced per iteration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14241) Fix deadlock during cluster shutdown due to concurrent connection close
[
https://issues.apache.org/jira/browse/HBASE-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703212#comment-14703212
]
Ted Yu commented on HBASE-14241:
From https://builds.apache.org/job/PreCommit-HBASE-Build/15167/consoleFull :
{code}
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:713)
{code}
There were 82 occurrences of 'unable to create new native thread' on the above
page.
Fix deadlock during cluster shutdown due to concurrent connection close
---
Key: HBASE-14241
URL: https://issues.apache.org/jira/browse/HBASE-14241
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.2
Reporter: Andrew Purtell
Assignee: Ted Yu
Priority: Critical
Fix For: 2.0.0, 1.0.2, 1.2.0, 1.3.0, 1.1.3
Attachments: 14241-v2.txt, 14241-v3.txt, 14241-v4.txt, 14241-v5.txt,
deadlock.txt.gz
Caught while testing branch-1.0, shutting down TestMasterMetricsWrapper.
Found one Java-level deadlock:
=
MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0:
waiting to lock monitor 0x7f2a040051c8 (object 0x0007e36108a8, a
org.apache.hadoop.hbase.util.PoolMap),
which is held by M:0;ip-10-32-130-237:55342
M:0;ip-10-32-130-237:55342:
waiting to lock monitor 0x7f2a04005118 (object 0x0007e3610b00, a
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),
which is held by MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0
Full stack dump and deadlock debug output attached.
Root cause:
In RpcClientImpl#close(), we obtain lock on connections first:
{code}
synchronized (connections) {
for (Connection conn : connections.values()) {
{code}
Then markClosed() tries to obtain lock on connection object:
{code}
if (!conn.isAlive()) {
conn.markClosed(new InterruptedIOException(RpcClient is closing));
conn.close();
{code}
Another thread, MetaServerShutdownHandler, calls
RpcClientImpl$Connection#setupIOstreams() where :
{code}
markClosed(e);
close();
{code}
Lock on connection object is obtained first, then lock on connections is
attempted, leading to deadlock:
{code}
synchronized (connections) {
connections.removeValue(remoteId, this);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14241) Fix deadlock during cluster shutdown due to concurrent connection close
[
https://issues.apache.org/jira/browse/HBASE-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14241:
---
Description:
Caught while testing branch-1.0, shutting down TestMasterMetricsWrapper.
Found one Java-level deadlock:
=
MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0:
waiting to lock monitor 0x7f2a040051c8 (object 0x0007e36108a8, a
org.apache.hadoop.hbase.util.PoolMap),
which is held by M:0;ip-10-32-130-237:55342
M:0;ip-10-32-130-237:55342:
waiting to lock monitor 0x7f2a04005118 (object 0x0007e3610b00, a
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),
which is held by MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0
Full stack dump and deadlock debug output attached.
Root cause:
In RpcClientImpl#close(), we obtain lock on connections first:
{code}
synchronized (connections) {
for (Connection conn : connections.values()) {
{code}
Then markClosed() tries to obtain lock on connection object:
{code}
if (!conn.isAlive()) {
conn.markClosed(new InterruptedIOException(RpcClient is closing));
conn.close();
{code}
Another thread, MetaServerShutdownHandler, calls
RpcClientImpl$Connection#setupIOstreams() where :
{code}
markClosed(e);
close();
{code}
Lock on connection object is obtained first, then lock on connections is
attempted, leading to deadlock:
{code}
synchronized (connections) {
connections.removeValue(remoteId, this);
}
{code}
was:
Caught while testing branch-1.0, shutting down TestMasterMetricsWrapper.
Found one Java-level deadlock:
=
MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0:
waiting to lock monitor 0x7f2a040051c8 (object 0x0007e36108a8, a
org.apache.hadoop.hbase.util.PoolMap),
which is held by M:0;ip-10-32-130-237:55342
M:0;ip-10-32-130-237:55342:
waiting to lock monitor 0x7f2a04005118 (object 0x0007e3610b00, a
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),
which is held by MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0
Full stack dump and deadlock debug output attached.
Fix deadlock during cluster shutdown due to concurrent connection close
---
Key: HBASE-14241
URL: https://issues.apache.org/jira/browse/HBASE-14241
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.2
Reporter: Andrew Purtell
Assignee: Ted Yu
Priority: Critical
Fix For: 2.0.0, 1.0.2, 1.2.0, 1.3.0, 1.1.3
Attachments: 14241-v2.txt, 14241-v3.txt, 14241-v4.txt, 14241-v5.txt,
deadlock.txt.gz
Caught while testing branch-1.0, shutting down TestMasterMetricsWrapper.
Found one Java-level deadlock:
=
MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0:
waiting to lock monitor 0x7f2a040051c8 (object 0x0007e36108a8, a
org.apache.hadoop.hbase.util.PoolMap),
which is held by M:0;ip-10-32-130-237:55342
M:0;ip-10-32-130-237:55342:
waiting to lock monitor 0x7f2a04005118 (object 0x0007e3610b00, a
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),
which is held by MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0
Full stack dump and deadlock debug output attached.
Root cause:
In RpcClientImpl#close(), we obtain lock on connections first:
{code}
synchronized (connections) {
for (Connection conn : connections.values()) {
{code}
Then markClosed() tries to obtain lock on connection object:
{code}
if (!conn.isAlive()) {
conn.markClosed(new InterruptedIOException(RpcClient is closing));
conn.close();
{code}
Another thread, MetaServerShutdownHandler, calls
RpcClientImpl$Connection#setupIOstreams() where :
{code}
markClosed(e);
close();
{code}
Lock on connection object is obtained first, then lock on connections is
attempted, leading to deadlock:
{code}
synchronized (connections) {
connections.removeValue(remoteId, this);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14215) Default cost used for PrimaryRegionCountSkewCostFunction is not sufficient
[
https://issues.apache.org/jira/browse/HBASE-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703160#comment-14703160
]
Biju Nair commented on HBASE-14215:
---
Thanks all for the comments. The following is the reasoning for setting the
value to 1 for {{hbase.master.balancer.stochastic.primaryRegionCountCost}}
in the context of region replication 1 based on my limited understanding.
- High value of 10 set to {{regionReplicaHostCostKey}} helps with
reducing/eliminating duplicate replication of regions on hosts and in turn
improving availability. Elimination of duplicate regions on hosts also helps
with performance with secondary calls being made to different hosts and hence
distributing the query load.
- The function to reduce duplicates of region replicas on the same rack which
uses the multiplier {{regionReplicaRackCostKey}} helps with availability but
not as much with the performance of queries since they get distributed to the
servers with no consideration to rack.
- The new function to reduce skews of primary region replicas on servers is to
distribute the primaries uniformly across all the servers which intern
distributes query load and improves performance since by default all queries
will get serviced by primary replicas.
While duplicate replicas on servers are eliminated by high cost of 10 which
also helps with performance, the next criteria was to balance between rack
level availability vs request performance. By setting
{{primaryRegionCountCost}} equal to {{regionReplicaRackCostKey}} which is
1 the assumption was that the candidate cluster which will be used will be
balanced for availability and performance. Let me know what was overlooked so
it will help with the understanding.
As suggested will try other cost values and update the ticket. Currently we
are using site.xml to vary the costs.
Default cost used for PrimaryRegionCountSkewCostFunction is not sufficient
---
Key: HBASE-14215
URL: https://issues.apache.org/jira/browse/HBASE-14215
Project: HBase
Issue Type: Bug
Components: Balancer
Reporter: Biju Nair
Priority: Minor
Attachments: 14215-v1.txt
Current multiplier of 500 used in the stochastic balancer cost function
{{PrimaryRegionCountSkewCostFunction}} to calculate the cost of total
primary replication skew doesn't seem to be sufficient to prevent the skews
(Refer HBASE-14110). We would want the default cost to be a higher value so
that skews in primary region replica has higher cost. The following is the
test result by setting the multiplier value to 1 (same as the region
replica rack cost multiplier) on a 3 Rack 9 RS node cluster which seems to
get the balancer distribute the primaries uniformly.
*Initial Primary replica distribution - using the current multiplier*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9 124
r3n10 135
r3n11 124
r3n9129
*After long duration of read writes - using current multiplier*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9124
r3n10 135
r3n11 124
r3n9129
*After manual balancing*
r1n10 102
r1n11 85
r1n988
r2n10 120
r2n11 120
r2n9124
r3n10 135
r3n11 124
r3n9129
*Increased multiplier for primaryRegionCountSkewCost to 1*
r1n10 114
r1n11 113
r1n9114
r2n10 114
r2n11 114
r2n9113
r3n10 115
r3n11 115
r3n9115
Setting the {{PrimaryRegionCountSkewCostFunction}} multiplier value to 1
should help HBase general use.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14256) Flush task message may be confusing when region is recovered
Lars George created HBASE-14256:
---
Summary: Flush task message may be confusing when region is
recovered
Key: HBASE-14256
URL: https://issues.apache.org/jira/browse/HBASE-14256
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.2.0
Reporter: Lars George
In {{HRegion.setRecovering()}} we have this code:
{code}
// force a flush only if region replication is set up for this region.
Otherwise no need.
boolean forceFlush = getTableDesc().getRegionReplication() 1;
// force a flush first
MonitoredTask status = TaskMonitor.get().createStatus(
Flushing region + this + because recovery is finished);
try {
if (forceFlush) {
internalFlushcache(status);
}
{code}
So we only optionally force flush after a recovery of a region, but the message
always is set to Flushing..., which might be confusing. We should change the
message based on {{forceFlush}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14210) Create test for cell level ACLs involving user group
[ https://issues.apache.org/jira/browse/HBASE-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703228#comment-14703228 ] Hudson commented on HBASE-14210: ABORTED: Integrated in HBase-TRUNK #6736 (See [https://builds.apache.org/job/HBase-TRUNK/6736/]) HBASE-14210 Create test for cell level ACLs involving user group (apurtell: rev 1bb9e3ae966853db8c4138b4abbe14636d7592db) * hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLs.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLWithMultipleVersions.java Create test for cell level ACLs involving user group Key: HBASE-14210 URL: https://issues.apache.org/jira/browse/HBASE-14210 Project: HBase Issue Type: Test Reporter: Ted Yu Assignee: Ashish Singhi Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.15, 1.0.3, 1.1.3 Attachments: HBASE-14210-0.98.patch, HBASE-14210-branch-1.patch, HBASE-14210-v1.patch, HBASE-14210-v2.patch, HBASE-14210-v3.patch, HBASE-14210.patch Currently we have TestCellACLs and TestCellACLWithMultipleVersions which exercise cell level ACLs for users. However, test for cell level ACLs involving user group is missing. This issue is to add such test(s) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14210) Create test for cell level ACLs involving user group
[ https://issues.apache.org/jira/browse/HBASE-14210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703237#comment-14703237 ] Hudson commented on HBASE-14210: ABORTED: Integrated in HBase-1.3 #116 (See [https://builds.apache.org/job/HBase-1.3/116/]) HBASE-14210 Create test for cell level ACLs involving user group (apurtell: rev 22d39c0712fdaf7affda732a8214e24385eebb98) * hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLWithMultipleVersions.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestCellACLs.java Create test for cell level ACLs involving user group Key: HBASE-14210 URL: https://issues.apache.org/jira/browse/HBASE-14210 Project: HBase Issue Type: Test Reporter: Ted Yu Assignee: Ashish Singhi Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.15, 1.0.3, 1.1.3 Attachments: HBASE-14210-0.98.patch, HBASE-14210-branch-1.patch, HBASE-14210-v1.patch, HBASE-14210-v2.patch, HBASE-14210-v3.patch, HBASE-14210.patch Currently we have TestCellACLs and TestCellACLWithMultipleVersions which exercise cell level ACLs for users. However, test for cell level ACLs involving user group is missing. This issue is to add such test(s) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14255) Simplify Cell creation post 1.0
[
https://issues.apache.org/jira/browse/HBASE-14255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14255:
--
Priority: Critical (was: Major)
Simplify Cell creation post 1.0
---
Key: HBASE-14255
URL: https://issues.apache.org/jira/browse/HBASE-14255
Project: HBase
Issue Type: Improvement
Components: Client
Affects Versions: 1.0.0, 2.0.0
Reporter: Lars George
Priority: Critical
After the switch to the new Cell based client API, and making KeyValue
private (but especially as soon as DBB backed Cells land) it is rather
difficult to create a {{Cell}} instance. I am using this now:
{code}
@Override
public void postGetOp(ObserverContextRegionCoprocessorEnvironment e,
Get get, ListCell results) throws IOException {
Put put = new Put(get.getRow());
put.addColumn(get.getRow(), FIXED_COLUMN, Bytes.toBytes(counter.get()));
CellScanner scanner = put.cellScanner();
scanner.advance();
Cell cell = scanner.current();
LOG.debug(Adding fake cell: + cell);
results.add(cell);
}
{code}
That is, I have to create a {{Put}} instance to add a cell and then retrieve
its instance. The {{KeyValue}} methods are private now and should not be
used. Create a CellBuilder helper?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14255) Simplify Cell creation post 1.0
Lars George created HBASE-14255:
---
Summary: Simplify Cell creation post 1.0
Key: HBASE-14255
URL: https://issues.apache.org/jira/browse/HBASE-14255
Project: HBase
Issue Type: Improvement
Components: Client
Affects Versions: 1.0.0, 2.0.0
Reporter: Lars George
After the switch to the new Cell based client API, and making KeyValue private
(but especially as soon as DBB backed Cells land) it is rather difficult to
create a {{Cell}} instance. I am using this now:
{code}
@Override
public void postGetOp(ObserverContextRegionCoprocessorEnvironment e,
Get get, ListCell results) throws IOException {
Put put = new Put(get.getRow());
put.addColumn(get.getRow(), FIXED_COLUMN, Bytes.toBytes(counter.get()));
CellScanner scanner = put.cellScanner();
scanner.advance();
Cell cell = scanner.current();
LOG.debug(Adding fake cell: + cell);
results.add(cell);
}
{code}
That is, I have to create a {{Put}} instance to add a cell and then retrieve
its instance. The {{KeyValue}} methods are private now and should not be used.
Create a CellBuilder helper?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14241) Fix deadlock during cluster shutdown due to concurrent connection close
[
https://issues.apache.org/jira/browse/HBASE-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703173#comment-14703173
]
Hadoop QA commented on HBASE-14241:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12751257/14241-v5.txt
against master branch at commit 1bb9e3ae966853db8c4138b4abbe14636d7592db.
ATTACHMENT ID: 12751257
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestSCVFWithMiniCluster
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan2
org.apache.hadoop.hbase.TestAcidGuarantees
org.apache.hadoop.hbase.mapreduce.TestTableSnapshotInputFormat
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1
org.apache.hadoop.hbase.TestMetaTableAccessor
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDistributedLogReplay
org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper
org.apache.hadoop.hbase.TestFullLogReconstruction
org.apache.hadoop.hbase.regionserver.TestRowTooBig
org.apache.hadoop.hbase.mapreduce.TestImportTSVWithVisibilityLabels
org.apache.hadoop.hbase.mapreduce.TestCellCounter
org.apache.hadoop.hbase.mob.compactions.TestMobCompactor
org.apache.hadoop.hbase.mapreduce.TestImportTSVWithTTLs
org.apache.hadoop.hbase.mapreduce.TestWALRecordReader
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithACL
org.apache.hadoop.hbase.mapreduce.TestWALPlayer
org.apache.hadoop.hbase.mapreduce.TestImportExport
org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint
org.apache.hadoop.hbase.mapreduce.TestSyncTable
org.apache.hadoop.hbase.mob.compactions.TestPartitionedMobCompactor
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.regionserver.TestSplitWalDataLoss
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
org.apache.hadoop.hbase.mapreduce.TestImportTSVWithOperationAttributes
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.mapreduce.TestHashTable
org.apache.hadoop.hbase.mapreduce.TestHRegionPartitioner
org.apache.hadoop.hbase.mapreduce.TestImportTsv
{color:red}-1 core zombie tests{color}. There are 12 zombie test(s):
at
org.apache.hadoop.hbase.TestAcidGuarantees.testGetAtomicity(TestAcidGuarantees.java:362)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15167//testReport/
Release Findbugs (version 2.0.3)
[jira] [Updated] (HBASE-13127) Add timeouts on all tests so less zombie sightings
[ https://issues.apache.org/jira/browse/HBASE-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13127: -- Attachment: 13127.alternate.v7.txt Add timeouts on all tests so less zombie sightings -- Key: HBASE-13127 URL: https://issues.apache.org/jira/browse/HBASE-13127 Project: HBase Issue Type: Improvement Components: test Reporter: stack Assignee: stack Attachments: 13127.alternate.txt, 13127.alternate.txt, 13127.alternate.txt, 13127.alternate.txt, 13127.alternate.v2.txt, 13127.alternate.v3.txt, 13127.alternate.v3.txt, 13127.alternate.v3.txt, 13127.alternate.v3.txt, 13127.alternate.v4.txt, 13127.alternate.v5.txt, 13127.alternate.v6.txt, 13127.alternate.v6.txt, 13127.alternate.v7.txt, 13127.alternate.v7.txt, 13127.alternate.v7.txt, 13127.alternate.v7.txt, 13127.txt, 13127v2.txt [~Apache9] and [~octo47] have been working hard at trying to get our builds passing again. They are almost there. TRUNK just failed with a zombie TestMasterObserver. Help the lads out by adding timeouts on all tests so less zombie incidence... will help identify the frequent failing issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703356#comment-14703356
]
Sean Busbey commented on HBASE-14249:
-
won't that be an additional module? persuming it is, it'll just list
dependencies on the needed test-jars and get built at the normal compile /
package phases without its own test-jar.
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14239) Branch-1.2 AM can get stuck when meta moves
[
https://issues.apache.org/jira/browse/HBASE-14239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703368#comment-14703368
]
Elliott Clark commented on HBASE-14239:
---
I tested the v2 patch last night and things are better but not great yet.
I'm still getting some of these:
{code}
2015-08-19T09:19:24 Region server
hbase516.ash1.facebook.com,16020,1439963654848 reported a fatal error:
ABORTING region server hbase516.ash1.facebook.com,16020,1439963654848:
Exception running postOpenDeployTasks; region=142593938bea8fb99cb265258c19dfa1
Cause:
java.io.IOException: Failed to report opened region to master:
IntegegrationTestBigLinkedList.4,_\xCB%\xFC\xB2_\xCB
,1439965968754.142593938bea8fb99cb265258c19dfa1.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1927)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:327)
{code}
I have all the regionserver set to restart so it wasn't too big a deal. However
for anyone without the auto-restart it would have been enough deaths to kill
the test.
Branch-1.2 AM can get stuck when meta moves
---
Key: HBASE-14239
URL: https://issues.apache.org/jira/browse/HBASE-14239
Project: HBase
Issue Type: Sub-task
Components: Region Assignment
Affects Versions: 1.2.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 1.2.0
Attachments: HBASE-14239-v1.patch, HBASE-14239.patch
When regions are moving master can get totally stuck trying to talk to meta.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13376) Improvements to Stochastic load balancer
[ https://issues.apache.org/jira/browse/HBASE-13376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703394#comment-14703394 ] Hudson commented on HBASE-13376: SUCCESS: Integrated in HBase-0.98 #1088 (See [https://builds.apache.org/job/HBase-0.98/1088/]) HBASE-13376 Improvements to Stochastic load balancer (Vandana Ayyalasomayajula) (tedyu: rev 2fd9c8df3b261dd596f68fff1c09c803da1ccdc3) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/RegionLocationFinder.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java Improvements to Stochastic load balancer Key: HBASE-13376 URL: https://issues.apache.org/jira/browse/HBASE-13376 Project: HBase Issue Type: Improvement Components: Balancer Affects Versions: 1.0.0, 0.98.12 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Fix For: 2.0.0, 0.98.14, 1.3.0 Attachments: 13376-v2.txt, 13376-v5.patch, 13376_4.patch, HBASE-13376.patch, HBASE-13376_0.98.txt, HBASE-13376_0.98_v2.patch, HBASE-13376_0.txt, HBASE-13376_1.txt, HBASE-13376_1_1.txt, HBASE-13376_2.patch, HBASE-13376_2_branch-1.patch, HBASE-13376_3.patch, HBASE-13376_3.patch, HBASE-13376_4.patch, HBASE-13376_5_branch-1.patch, HBASE-13376_6_branch-1.patch, HBASE-13376_98.patch, HBASE-13376_branch-1.patch, HBASE-13376_v3_0.98.patch, HBASE-13376_v4_0.98.patch There are two things this jira tries to address: 1. The locality picker in the stochastic balancer does not pick regions with least locality as candidates for swap/move. So when any user configures locality cost in the configs, the balancer does not always seems to move regions with bad locality. 2. When a cluster has equal number of loaded regions, it always picks the first one. It should pick a random region on one of the equally loaded servers. This improves a chance of finding a good candidate, when load picker is invoked several times. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13480) ShortCircuitConnection doesn't short-circuit all calls as expected
[
https://issues.apache.org/jira/browse/HBASE-13480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703472#comment-14703472
]
Lars Hofhansl commented on HBASE-13480:
---
It's technically OK. At the same time we do not have go out of our way to break
APIs :)
ShortCircuitConnection doesn't short-circuit all calls as expected
--
Key: HBASE-13480
URL: https://issues.apache.org/jira/browse/HBASE-13480
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: Josh Elser
Assignee: Jingcheng Du
Fix For: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.3
Attachments: HBASE-13480.patch
Noticed the following situation in debugging unexpected unit tests failures
in HBASE-13351.
{{ConnectionUtils#createShortCircuitHConnection(Connection, ServerName,
AdminService.BlockingInterface, ClientService.BlockingInterface)}} is
intended to avoid the extra RPC by calling the server's instantiation of the
protobuf rpc stub directly for the AdminService and ClientService.
The problem is that this is insufficient to actually avoid extra remote
RPCs as all other calls to the Connection are routed to a real Connection
instance. As such, any object created by the real Connection (such as an
HTable) will use the real Connection, not the SSC.
The end result is that
{{MasterRpcService#reportRegionStateTransition(RpcController,
ReportRegionStateTransitionRequest)}} will make additional remote RPCs over
what it thinks is an SSC through a {{Get}} on {{HTable}} which was
constructed using the SSC, but the {{Get}} itself will use the underlying
real Connection instead of the SSC. With insufficiently sized thread pools,
this has been observed to result in RPC deadlock in the HMaster where an RPC
attempts to make another RPC but there are no more threads available to
service the second RPC so the first RPC blocks indefinitely.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703281#comment-14703281
]
Elliott Clark commented on HBASE-14249:
---
We are going to want to create a shaded test jar at some point for people that
are trying to use the test-util jar and need the relocations.
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14234) Exception encountered in WALProcedureStore#rollWriter() should be properly handled
[
https://issues.apache.org/jira/browse/HBASE-14234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14234:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Exception encountered in WALProcedureStore#rollWriter() should be properly
handled
--
Key: HBASE-14234
URL: https://issues.apache.org/jira/browse/HBASE-14234
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.0.0, 1.2.0, 1.3.0
Attachments: 14234-v1.txt
Observed the following in recent Jenkins build
(https://builds.apache.org/job/HBase-TRUNK/6732/console):
{code}
testWALfencingWithoutWALRolling(org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures)
Time elapsed: 9.938 sec ERROR!
org.apache.hadoop.ipc.RemoteException: failed to create file
/user/jenkins/test-data/0d9e3047-6bb1-4219-9ed2-5b9884176321/MasterProcWALs/state-0002.log
for DFSClient_NONMAPREDUCE_-966558185_1 for client 127.0.0.1 because current
leaseholder is trying to recreate file.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:2589)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2386)
{code}
When file creation fails (e.g. due to RemoteException), we should handle the
exception by returning false.
Similar handling can be applied to failure in writing header.
Thanks to [~mbertozzi] for offline discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14230) replace reflection in FSHlog with HdfsDataOutputStream#getCurrentBlockReplication()
[
https://issues.apache.org/jira/browse/HBASE-14230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703324#comment-14703324
]
Ted Yu commented on HBASE-14230:
In https://builds.apache.org/job/PreCommit-HBASE-Build/15166/consoleFull, you
can see 66 occurrences of 'unable to create new native thread'
replace reflection in FSHlog with
HdfsDataOutputStream#getCurrentBlockReplication()
---
Key: HBASE-14230
URL: https://issues.apache.org/jira/browse/HBASE-14230
Project: HBase
Issue Type: Improvement
Components: wal
Reporter: Heng Chen
Priority: Minor
Attachments: HBASE-14230.patch
As comment TODO said, we use
{{HdfsDataOutputStream#getCurrentBlockReplication}} and
{{DFSOutputStream.getPipeLine}} to replace reflection in FSHlog
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14241) Fix deadlock during cluster shutdown due to concurrent connection close
[
https://issues.apache.org/jira/browse/HBASE-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703421#comment-14703421
]
Hudson commented on HBASE-14241:
FAILURE: Integrated in HBase-TRUNK #6737 (See
[https://builds.apache.org/job/HBase-TRUNK/6737/])
HBASE-14241 Fix deadlock during cluster shutdown due to concurrent connection
close (tedyu: rev 16f8d277088987dc8d4bb1614d05ce712bfc245d)
* hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java
Fix deadlock during cluster shutdown due to concurrent connection close
---
Key: HBASE-14241
URL: https://issues.apache.org/jira/browse/HBASE-14241
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.2
Reporter: Andrew Purtell
Assignee: Ted Yu
Priority: Critical
Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.2, 1.3.0
Attachments: 14241-v2.txt, 14241-v3.txt, 14241-v4.txt, 14241-v5.txt,
deadlock.txt.gz
Caught while testing branch-1.0, shutting down TestMasterMetricsWrapper.
Found one Java-level deadlock:
=
MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0:
waiting to lock monitor 0x7f2a040051c8 (object 0x0007e36108a8, a
org.apache.hadoop.hbase.util.PoolMap),
which is held by M:0;ip-10-32-130-237:55342
M:0;ip-10-32-130-237:55342:
waiting to lock monitor 0x7f2a04005118 (object 0x0007e3610b00, a
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),
which is held by MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0
Full stack dump and deadlock debug output attached.
Root cause:
In RpcClientImpl#close(), we obtain lock on connections first:
{code}
synchronized (connections) {
for (Connection conn : connections.values()) {
{code}
Then markClosed() tries to obtain lock on connection object:
{code}
if (!conn.isAlive()) {
conn.markClosed(new InterruptedIOException(RpcClient is closing));
conn.close();
{code}
Another thread, MetaServerShutdownHandler, calls
RpcClientImpl$Connection#setupIOstreams() where :
{code}
markClosed(e);
close();
{code}
Lock on connection object is obtained first, then lock on connections is
attempted, leading to deadlock:
{code}
synchronized (connections) {
connections.removeValue(remoteId, this);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14239) Branch-1.2 AM can get stuck when meta moves
[
https://issues.apache.org/jira/browse/HBASE-14239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703368#comment-14703368
]
Elliott Clark edited comment on HBASE-14239 at 8/19/15 5:03 PM:
I tested the v2 patch last night and things are better but not great yet.
I'm still getting some of these:
{code}
2015-08-19T09:19:24 Region server
hbase516.ash1.facebook.com,16020,1439963654848 reported a fatal error:
ABORTING region server hbase516.ash1.facebook.com,16020,1439963654848:
Exception running postOpenDeployTasks; region=142593938bea8fb99cb265258c19dfa1
Cause:
java.io.IOException: Failed to report opened region to master:
IntegegrationTestBigLinkedList.4,_\xCB%\xFC\xB2_\xCB
,1439965968754.142593938bea8fb99cb265258c19dfa1.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1927)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:327)
{code}
I have all the regionservers set to restart so it wasn't too big a deal.
However for anyone without the auto-restart it would have been enough deaths to
kill the test.
was (Author: eclark):
I tested the v2 patch last night and things are better but not great yet.
I'm still getting some of these:
{code}
2015-08-19T09:19:24 Region server
hbase516.ash1.facebook.com,16020,1439963654848 reported a fatal error:
ABORTING region server hbase516.ash1.facebook.com,16020,1439963654848:
Exception running postOpenDeployTasks; region=142593938bea8fb99cb265258c19dfa1
Cause:
java.io.IOException: Failed to report opened region to master:
IntegegrationTestBigLinkedList.4,_\xCB%\xFC\xB2_\xCB
,1439965968754.142593938bea8fb99cb265258c19dfa1.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1927)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:327)
{code}
I have all the regionserver set to restart so it wasn't too big a deal. However
for anyone without the auto-restart it would have been enough deaths to kill
the test.
Branch-1.2 AM can get stuck when meta moves
---
Key: HBASE-14239
URL: https://issues.apache.org/jira/browse/HBASE-14239
Project: HBase
Issue Type: Sub-task
Components: Region Assignment
Affects Versions: 1.2.0
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 1.2.0
Attachments: HBASE-14239-v1.patch, HBASE-14239.patch
When regions are moving master can get totally stuck trying to talk to meta.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14258) Make region_mover.rb script case insensitive in regard to hostname
[ https://issues.apache.org/jira/browse/HBASE-14258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14258: -- Fix Version/s: 2.0.0 Make region_mover.rb script case insensitive in regard to hostname -- Key: HBASE-14258 URL: https://issues.apache.org/jira/browse/HBASE-14258 Project: HBase Issue Type: Bug Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Fix For: 2.0.0 The script is case sensitive and fails when case of a host name being unloaded does not match with a case of a region server name returned by HBase API. This doc clarifies IETF rules on case insensitivities in DNS: https://www.ietf.org/rfc/rfc4343.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-14251) javadoc jars use LICENSE/NOTICE from primary artifact
[ https://issues.apache.org/jira/browse/HBASE-14251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-14251 started by Sean Busbey. --- javadoc jars use LICENSE/NOTICE from primary artifact - Key: HBASE-14251 URL: https://issues.apache.org/jira/browse/HBASE-14251 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.2.0, 1.1.2, 0.98.15, 1.0.3 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker Our generated javadoc jars have the same LICENSE/NOTICE files as our primary artifacts but do not include a copy of hte full source. the following modules end up with incorrect artifacts: * hbase-server * hbase-common (maybe? depends on the are-apis-copyrightable court case) * hbase-thrift -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14258) Make region_mover.rb script case insensitive in regard to hostname
[ https://issues.apache.org/jira/browse/HBASE-14258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14258: -- Priority: Minor (was: Major) Make region_mover.rb script case insensitive in regard to hostname -- Key: HBASE-14258 URL: https://issues.apache.org/jira/browse/HBASE-14258 Project: HBase Issue Type: Bug Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Priority: Minor Fix For: 2.0.0 Attachments: HBASE-14258.patch The script is case sensitive and fails when case of a host name being unloaded does not match with a case of a region server name returned by HBase API. This doc clarifies IETF rules on case insensitivities in DNS: https://www.ietf.org/rfc/rfc4343.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14251) javadoc jars use LICENSE/NOTICE from primary artifact
[ https://issues.apache.org/jira/browse/HBASE-14251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703393#comment-14703393 ] Sean Busbey commented on HBASE-14251: - I think I can solve this using the javadoc resources directory. Just have to remember how to build the javadoc jars. :/ javadoc jars use LICENSE/NOTICE from primary artifact - Key: HBASE-14251 URL: https://issues.apache.org/jira/browse/HBASE-14251 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.2.0, 1.1.2, 0.98.15, 1.0.3 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker Our generated javadoc jars have the same LICENSE/NOTICE files as our primary artifacts but do not include a copy of hte full source. the following modules end up with incorrect artifacts: * hbase-server * hbase-common (maybe? depends on the are-apis-copyrightable court case) * hbase-thrift -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703440#comment-14703440
]
Elliott Clark commented on HBASE-14249:
---
It would be a different module, though I'm not sure about the phases. I've had
issues with getting everything to pull to compile phase sometimes.
But whatever is needed so that rc's can pass for now.
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14241) Fix deadlock during cluster shutdown due to concurrent connection close
[
https://issues.apache.org/jira/browse/HBASE-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703477#comment-14703477
]
Hudson commented on HBASE-14241:
SUCCESS: Integrated in HBase-1.2-IT #99 (See
[https://builds.apache.org/job/HBase-1.2-IT/99/])
HBASE-14241 Fix deadlock during cluster shutdown due to concurrent connection
close (tedyu: rev 639018a857a5e58f56d1db45e3f2d0e6043e2650)
* hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java
Fix deadlock during cluster shutdown due to concurrent connection close
---
Key: HBASE-14241
URL: https://issues.apache.org/jira/browse/HBASE-14241
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.2
Reporter: Andrew Purtell
Assignee: Ted Yu
Priority: Critical
Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.2, 1.3.0
Attachments: 14241-v2.txt, 14241-v3.txt, 14241-v4.txt, 14241-v5.txt,
deadlock.txt.gz
Caught while testing branch-1.0, shutting down TestMasterMetricsWrapper.
Found one Java-level deadlock:
=
MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0:
waiting to lock monitor 0x7f2a040051c8 (object 0x0007e36108a8, a
org.apache.hadoop.hbase.util.PoolMap),
which is held by M:0;ip-10-32-130-237:55342
M:0;ip-10-32-130-237:55342:
waiting to lock monitor 0x7f2a04005118 (object 0x0007e3610b00, a
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),
which is held by MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0
Full stack dump and deadlock debug output attached.
Root cause:
In RpcClientImpl#close(), we obtain lock on connections first:
{code}
synchronized (connections) {
for (Connection conn : connections.values()) {
{code}
Then markClosed() tries to obtain lock on connection object:
{code}
if (!conn.isAlive()) {
conn.markClosed(new InterruptedIOException(RpcClient is closing));
conn.close();
{code}
Another thread, MetaServerShutdownHandler, calls
RpcClientImpl$Connection#setupIOstreams() where :
{code}
markClosed(e);
close();
{code}
Lock on connection object is obtained first, then lock on connections is
attempted, leading to deadlock:
{code}
synchronized (connections) {
connections.removeValue(remoteId, this);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13480) ShortCircuitConnection doesn't short-circuit all calls as expected
[
https://issues.apache.org/jira/browse/HBASE-13480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703488#comment-14703488
]
Jesse Yates commented on HBASE-13480:
-
Does marking it deprecated and point ppl to the new method make everyone happy
then? :)
ShortCircuitConnection doesn't short-circuit all calls as expected
--
Key: HBASE-13480
URL: https://issues.apache.org/jira/browse/HBASE-13480
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: Josh Elser
Assignee: Jingcheng Du
Fix For: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.3
Attachments: HBASE-13480.patch
Noticed the following situation in debugging unexpected unit tests failures
in HBASE-13351.
{{ConnectionUtils#createShortCircuitHConnection(Connection, ServerName,
AdminService.BlockingInterface, ClientService.BlockingInterface)}} is
intended to avoid the extra RPC by calling the server's instantiation of the
protobuf rpc stub directly for the AdminService and ClientService.
The problem is that this is insufficient to actually avoid extra remote
RPCs as all other calls to the Connection are routed to a real Connection
instance. As such, any object created by the real Connection (such as an
HTable) will use the real Connection, not the SSC.
The end result is that
{{MasterRpcService#reportRegionStateTransition(RpcController,
ReportRegionStateTransitionRequest)}} will make additional remote RPCs over
what it thinks is an SSC through a {{Get}} on {{HTable}} which was
constructed using the SSC, but the {{Get}} itself will use the underlying
real Connection instead of the SSC. With insufficiently sized thread pools,
this has been observed to result in RPC deadlock in the HMaster where an RPC
attempts to make another RPC but there are no more threads available to
service the second RPC so the first RPC blocks indefinitely.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14253) update docs + build for maven 3.0.4+
[
https://issues.apache.org/jira/browse/HBASE-14253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14253:
---
Assignee: Lars Francke (was: Andrew Purtell)
Go for it [~lars_francke]!
update docs + build for maven 3.0.4+
Key: HBASE-14253
URL: https://issues.apache.org/jira/browse/HBASE-14253
Project: HBase
Issue Type: Task
Components: build, documentation
Affects Versions: 2.0.0
Reporter: Sean Busbey
Assignee: Lars Francke
Labels: beginner
our new hbase-spark module raises our minimum maven version from 3.0.0
(though I've only tried 3.0.3) to 3.0.4:
{code}
[ERROR] Failed to execute goal
net.alchim31.maven:scala-maven-plugin:3.2.0:add-source (scala-compile-first)
on project hbase-spark: The plugin
net.alchim31.maven:scala-maven-plugin:3.2.0 requires Maven version 3.0.4 -
[Help 1]
{code}
Update the docs to call out 3.0.4 and add an enforcer rule so that this
failure can happen at the start of a build rather than 15 minutes in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14257) Periodic flusher only handles hbase:meta, not other system tables
Lars George created HBASE-14257:
---
Summary: Periodic flusher only handles hbase:meta, not other
system tables
Key: HBASE-14257
URL: https://issues.apache.org/jira/browse/HBASE-14257
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 2.0.0, 1.2.0
Reporter: Lars George
In {{HRegion.shouldFlush}} we have
{code}
long modifiedFlushCheckInterval = flushCheckInterval;
if (getRegionInfo().isMetaRegion()
getRegionInfo().getReplicaId() == HRegionInfo.DEFAULT_REPLICA_ID) {
modifiedFlushCheckInterval = META_CACHE_FLUSH_INTERVAL;
}
{code}
That method is called by the {{PeriodicMemstoreFlusher}} thread, and prefers
the {{hbase:meta}} only for faster flushing. It should be doing the same for
other system tables. I suggest to use {{HRI.isSystemTable()}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14258) Make region_mover.rb script case insensitive in regard to hostname
Vladimir Rodionov created HBASE-14258: - Summary: Make region_mover.rb script case insensitive in regard to hostname Key: HBASE-14258 URL: https://issues.apache.org/jira/browse/HBASE-14258 Project: HBase Issue Type: Bug Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov The script is case sensitive and fails when case of a host name being unloaded does not match with a case of a region server name returned by HBase API. This doc clarifies IETF rules on case insensitivities in DNS: https://www.ietf.org/rfc/rfc4343.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14258) Make region_mover.rb script case insensitive in regard to hostname
[ https://issues.apache.org/jira/browse/HBASE-14258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14258: -- Attachment: HBASE-14258.patch Patch for master. Make region_mover.rb script case insensitive in regard to hostname -- Key: HBASE-14258 URL: https://issues.apache.org/jira/browse/HBASE-14258 Project: HBase Issue Type: Bug Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Fix For: 2.0.0 Attachments: HBASE-14258.patch The script is case sensitive and fails when case of a host name being unloaded does not match with a case of a region server name returned by HBase API. This doc clarifies IETF rules on case insensitivities in DNS: https://www.ietf.org/rfc/rfc4343.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14258) Make region_mover.rb script case insensitive in regard to hostname
[ https://issues.apache.org/jira/browse/HBASE-14258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14258: -- Status: Patch Available (was: Open) Make region_mover.rb script case insensitive in regard to hostname -- Key: HBASE-14258 URL: https://issues.apache.org/jira/browse/HBASE-14258 Project: HBase Issue Type: Bug Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Fix For: 2.0.0 Attachments: HBASE-14258.patch The script is case sensitive and fails when case of a host name being unloaded does not match with a case of a region server name returned by HBase API. This doc clarifies IETF rules on case insensitivities in DNS: https://www.ietf.org/rfc/rfc4343.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14241) Fix deadlock during cluster shutdown due to concurrent connection close
[
https://issues.apache.org/jira/browse/HBASE-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703615#comment-14703615
]
Hudson commented on HBASE-14241:
FAILURE: Integrated in HBase-1.0 #1020 (See
[https://builds.apache.org/job/HBase-1.0/1020/])
HBASE-14241 Fix deadlock during cluster shutdown due to concurrent connection
close (tedyu: rev de5bdbe6b2e4e494f0ec8d06c1ab8cc51ee9c187)
* hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java
Fix deadlock during cluster shutdown due to concurrent connection close
---
Key: HBASE-14241
URL: https://issues.apache.org/jira/browse/HBASE-14241
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.2
Reporter: Andrew Purtell
Assignee: Ted Yu
Priority: Critical
Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.2, 1.3.0
Attachments: 14241-v2.txt, 14241-v3.txt, 14241-v4.txt, 14241-v5.txt,
deadlock.txt.gz
Caught while testing branch-1.0, shutting down TestMasterMetricsWrapper.
Found one Java-level deadlock:
=
MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0:
waiting to lock monitor 0x7f2a040051c8 (object 0x0007e36108a8, a
org.apache.hadoop.hbase.util.PoolMap),
which is held by M:0;ip-10-32-130-237:55342
M:0;ip-10-32-130-237:55342:
waiting to lock monitor 0x7f2a04005118 (object 0x0007e3610b00, a
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),
which is held by MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0
Full stack dump and deadlock debug output attached.
Root cause:
In RpcClientImpl#close(), we obtain lock on connections first:
{code}
synchronized (connections) {
for (Connection conn : connections.values()) {
{code}
Then markClosed() tries to obtain lock on connection object:
{code}
if (!conn.isAlive()) {
conn.markClosed(new InterruptedIOException(RpcClient is closing));
conn.close();
{code}
Another thread, MetaServerShutdownHandler, calls
RpcClientImpl$Connection#setupIOstreams() where :
{code}
markClosed(e);
close();
{code}
Lock on connection object is obtained first, then lock on connections is
attempted, leading to deadlock:
{code}
synchronized (connections) {
connections.removeValue(remoteId, this);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14259) Backport Namespace quota support to 98 branch
[ https://issues.apache.org/jira/browse/HBASE-14259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vandana Ayyalasomayajula updated HBASE-14259: - Attachment: HBASE-14259_v1_0.98.patch Backport Namespace quota support to 98 branch -- Key: HBASE-14259 URL: https://issues.apache.org/jira/browse/HBASE-14259 Project: HBase Issue Type: New Feature Affects Versions: 0.98.14 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: HBASE-14259_v1_0.98.patch Namespace quota support (HBASE-8410) has been backported to branch-1 (HBASE-13438). This jira would backport the same to 98 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14259) Backport Namespace quota support to 98 branch
[ https://issues.apache.org/jira/browse/HBASE-14259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vandana Ayyalasomayajula updated HBASE-14259: - Status: Patch Available (was: Open) Backport Namespace quota support to 98 branch -- Key: HBASE-14259 URL: https://issues.apache.org/jira/browse/HBASE-14259 Project: HBase Issue Type: New Feature Affects Versions: 0.98.14 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: HBASE-14259_v1_0.98.patch Namespace quota support (HBASE-8410) has been backported to branch-1 (HBASE-13438). This jira would backport the same to 98 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14259) Backport Namespace quota support to 98 branch
Vandana Ayyalasomayajula created HBASE-14259: Summary: Backport Namespace quota support to 98 branch Key: HBASE-14259 URL: https://issues.apache.org/jira/browse/HBASE-14259 Project: HBase Issue Type: New Feature Affects Versions: 0.98.14 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Namespace quota support (HBASE-8410) has been backported to branch-1 (HBASE-13438). This jira would backport the same to 98 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14259) Backport Namespace quota support to 98 branch
[ https://issues.apache.org/jira/browse/HBASE-14259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703661#comment-14703661 ] Vandana Ayyalasomayajula commented on HBASE-14259: -- [~apurtell] --Can you take a look at the patch ? Also, please point me to the compatibility checker tool that I need to run against the patch. Backport Namespace quota support to 98 branch -- Key: HBASE-14259 URL: https://issues.apache.org/jira/browse/HBASE-14259 Project: HBase Issue Type: New Feature Affects Versions: 0.98.14 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: HBASE-14259_v1_0.98.patch Namespace quota support (HBASE-8410) has been backported to branch-1 (HBASE-13438). This jira would backport the same to 98 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14241) Fix deadlock during cluster shutdown due to concurrent connection close
[
https://issues.apache.org/jira/browse/HBASE-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703596#comment-14703596
]
Hudson commented on HBASE-14241:
FAILURE: Integrated in HBase-1.3 #117 (See
[https://builds.apache.org/job/HBase-1.3/117/])
HBASE-14241 Fix deadlock during cluster shutdown due to concurrent connection
close (tedyu: rev 558aec84e835db6e3e7097dc472a816b62a72728)
* hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java
Fix deadlock during cluster shutdown due to concurrent connection close
---
Key: HBASE-14241
URL: https://issues.apache.org/jira/browse/HBASE-14241
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.2
Reporter: Andrew Purtell
Assignee: Ted Yu
Priority: Critical
Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.2, 1.3.0
Attachments: 14241-v2.txt, 14241-v3.txt, 14241-v4.txt, 14241-v5.txt,
deadlock.txt.gz
Caught while testing branch-1.0, shutting down TestMasterMetricsWrapper.
Found one Java-level deadlock:
=
MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0:
waiting to lock monitor 0x7f2a040051c8 (object 0x0007e36108a8, a
org.apache.hadoop.hbase.util.PoolMap),
which is held by M:0;ip-10-32-130-237:55342
M:0;ip-10-32-130-237:55342:
waiting to lock monitor 0x7f2a04005118 (object 0x0007e3610b00, a
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),
which is held by MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0
Full stack dump and deadlock debug output attached.
Root cause:
In RpcClientImpl#close(), we obtain lock on connections first:
{code}
synchronized (connections) {
for (Connection conn : connections.values()) {
{code}
Then markClosed() tries to obtain lock on connection object:
{code}
if (!conn.isAlive()) {
conn.markClosed(new InterruptedIOException(RpcClient is closing));
conn.close();
{code}
Another thread, MetaServerShutdownHandler, calls
RpcClientImpl$Connection#setupIOstreams() where :
{code}
markClosed(e);
close();
{code}
Lock on connection object is obtained first, then lock on connections is
attempted, leading to deadlock:
{code}
synchronized (connections) {
connections.removeValue(remoteId, this);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14241) Fix deadlock during cluster shutdown due to concurrent connection close
[
https://issues.apache.org/jira/browse/HBASE-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703635#comment-14703635
]
Hudson commented on HBASE-14241:
SUCCESS: Integrated in HBase-1.1 #623 (See
[https://builds.apache.org/job/HBase-1.1/623/])
HBASE-14241 Fix deadlock during cluster shutdown due to concurrent connection
close (tedyu: rev 062c69cef533e6f416cd6d01cf0565b5d3112cdf)
* hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java
Fix deadlock during cluster shutdown due to concurrent connection close
---
Key: HBASE-14241
URL: https://issues.apache.org/jira/browse/HBASE-14241
Project: HBase
Issue Type: Bug
Affects Versions: 1.0.2
Reporter: Andrew Purtell
Assignee: Ted Yu
Priority: Critical
Fix For: 2.0.0, 1.0.2, 1.2.0, 1.1.2, 1.3.0
Attachments: 14241-v2.txt, 14241-v3.txt, 14241-v4.txt, 14241-v5.txt,
deadlock.txt.gz
Caught while testing branch-1.0, shutting down TestMasterMetricsWrapper.
Found one Java-level deadlock:
=
MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0:
waiting to lock monitor 0x7f2a040051c8 (object 0x0007e36108a8, a
org.apache.hadoop.hbase.util.PoolMap),
which is held by M:0;ip-10-32-130-237:55342
M:0;ip-10-32-130-237:55342:
waiting to lock monitor 0x7f2a04005118 (object 0x0007e3610b00, a
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection),
which is held by MASTER_META_SERVER_OPERATIONS-ip-10-32-130-237:55342-0
Full stack dump and deadlock debug output attached.
Root cause:
In RpcClientImpl#close(), we obtain lock on connections first:
{code}
synchronized (connections) {
for (Connection conn : connections.values()) {
{code}
Then markClosed() tries to obtain lock on connection object:
{code}
if (!conn.isAlive()) {
conn.markClosed(new InterruptedIOException(RpcClient is closing));
conn.close();
{code}
Another thread, MetaServerShutdownHandler, calls
RpcClientImpl$Connection#setupIOstreams() where :
{code}
markClosed(e);
close();
{code}
Lock on connection object is obtained first, then lock on connections is
attempted, leading to deadlock:
{code}
synchronized (connections) {
connections.removeValue(remoteId, this);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14233) Branch-1.2 Merging regions can result in messed up tables
[
https://issues.apache.org/jira/browse/HBASE-14233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703674#comment-14703674
]
stack commented on HBASE-14233:
---
[~eclark] Master does not host meta in 1.2 (unless I have it wrong)
Branch-1.2 Merging regions can result in messed up tables
-
Key: HBASE-14233
URL: https://issues.apache.org/jira/browse/HBASE-14233
Project: HBase
Issue Type: Sub-task
Components: Region Assignment
Affects Versions: 1.2.0
Reporter: Elliott Clark
Priority: Critical
Fix For: 1.2.0
{code}
15/08/17 14:10:50 INFO master.RegionStates: Transition null to
{6486b9b9409b25f10eb806ec3bad442d state=MERGING_NEW, ts=1439845850422,
server=hbase508.ash1.facebook.com,16020,1439844470302}
15/08/17 14:10:57 INFO master.AssignmentManager: Failed to record merged
region 6486b9b9409b25f10eb806ec3bad442d
15/08/17 14:10:57 ERROR master.AssignmentManager: Failed to transtion region
from {6486b9b9409b25f10eb806ec3bad442d state=MERGING_NEW, ts=1439845850422,
server=hbase508.ash1.facebook.com,16020,1439844470302} to MERGE_PONR by
hbase508.ash1.facebook.com,16020,1439844470302: Failed to record the merging
in meta
15/08/17 14:11:08 WARN master.RegionStates: THIS SHOULD NOT HAPPEN:
unexpected {6486b9b9409b25f10eb806ec3bad442d state=MERGING_NEW,
ts=1439845857580, server=hbase508.ash1.facebook.com,16020,1439844470302}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14233) Branch-1.2 Merging regions can result in messed up tables
[
https://issues.apache.org/jira/browse/HBASE-14233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703653#comment-14703653
]
Elliott Clark commented on HBASE-14233:
---
I got this another time running the IT tests.
However I wonder if it's possible to keep this from happening when master hosts
meta? I've got another IT test running, but I'll try the theory in the coming
days.
Branch-1.2 Merging regions can result in messed up tables
-
Key: HBASE-14233
URL: https://issues.apache.org/jira/browse/HBASE-14233
Project: HBase
Issue Type: Sub-task
Components: Region Assignment
Affects Versions: 1.2.0
Reporter: Elliott Clark
Priority: Critical
Fix For: 1.2.0
{code}
15/08/17 14:10:50 INFO master.RegionStates: Transition null to
{6486b9b9409b25f10eb806ec3bad442d state=MERGING_NEW, ts=1439845850422,
server=hbase508.ash1.facebook.com,16020,1439844470302}
15/08/17 14:10:57 INFO master.AssignmentManager: Failed to record merged
region 6486b9b9409b25f10eb806ec3bad442d
15/08/17 14:10:57 ERROR master.AssignmentManager: Failed to transtion region
from {6486b9b9409b25f10eb806ec3bad442d state=MERGING_NEW, ts=1439845850422,
server=hbase508.ash1.facebook.com,16020,1439844470302} to MERGE_PONR by
hbase508.ash1.facebook.com,16020,1439844470302: Failed to record the merging
in meta
15/08/17 14:11:08 WARN master.RegionStates: THIS SHOULD NOT HAPPEN:
unexpected {6486b9b9409b25f10eb806ec3bad442d state=MERGING_NEW,
ts=1439845857580, server=hbase508.ash1.facebook.com,16020,1439844470302}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13919) Rationalize Client Timeout
[ https://issues.apache.org/jira/browse/HBASE-13919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703692#comment-14703692 ] Anil Gupta commented on HBASE-13919: Not exactly. Sometimes developers forget to put hbase-site.xml file of cluster in classpath of HBase Application. In that case, hbase client will assume that it needs to connect to a standalone instance. If the HBase application is trying to connect to table X. HBase app will try to connect to table X in standalone instance. This usually takes around 20 minutes to fail. Rationalize Client Timeout -- Key: HBASE-13919 URL: https://issues.apache.org/jira/browse/HBASE-13919 Project: HBase Issue Type: Brainstorming Reporter: Lars Hofhansl There are current many setting that influence how/when an HBase client times out. This is hard to configure, hard to understand, and badly documented. Instead there should as few tweaks as needed, and their setting should make sense. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12751) Allow RowLock to be reader writer
[
https://issues.apache.org/jira/browse/HBASE-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703746#comment-14703746
]
Hadoop QA commented on HBASE-12751:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12751285/HBASE-12751-v20.patch
against master branch at commit 16f8d277088987dc8d4bb1614d05ce712bfc245d.
ATTACHMENT ID: 12751285
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 81 new
or modified tests.
{color:red}-1 Anti-pattern{color}. The patch appears to
have anti-pattern where BYTES_COMPARATOR was omitted:
-getRegionInfo(), -1, new TreeMapbyte[], ListPath());.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationEndpoint
org.apache.hadoop.hbase.replication.TestPerTableCFReplication
org.apache.hadoop.hbase.wal.TestBoundedRegionGroupingProvider
org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpointNoMaster
org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.replication.TestMasterReplication
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster
org.apache.hadoop.hbase.regionserver.TestRegionFavoredNodes
org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager
org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable
org.apache.hadoop.hbase.TestGlobalMemStoreSize
org.apache.hadoop.hbase.wal.TestWALFiltering
org.apache.hadoop.hbase.replication.TestReplicationSmallTests
org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool
org.apache.hadoop.hbase.replication.TestReplicationWithTags
org.apache.hadoop.hbase.master.procedure.TestTruncateTableProcedure
org.apache.hadoop.hbase.replication.TestReplicationChangingPeerRegionservers
org.apache.hadoop.hbase.wal.TestDefaultWALProviderWithHLogKey
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.snapshot.TestMobRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure
org.apache.hadoop.hbase.wal.TestWALFactory
org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.master.procedure.TestAddColumnFamilyProcedure
org.apache.hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat
org.apache.hadoop.hbase.master.procedure.TestEnableTableProcedure
org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence
org.apache.hadoop.hbase.TestStochasticBalancerJmxMetrics
{color:red}-1 core zombie tests{color}. There are 16 zombie test(s):
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels.testVisibilityLabelsWithComplexLabels(TestVisibilityLabels.java:216)
at
[jira] [Updated] (HBASE-14165) The initial size of RWQueueRpcExecutor.queues should be (numWriteQueues + numReadQueues + numScanQueues)
[
https://issues.apache.org/jira/browse/HBASE-14165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14165:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 1.3.0
2.0.0
Status: Resolved (was: Patch Available)
Thanks for the patch, Jianwei.
Thanks for the reviews.
The initial size of RWQueueRpcExecutor.queues should be (numWriteQueues +
numReadQueues + numScanQueues)
-
Key: HBASE-14165
URL: https://issues.apache.org/jira/browse/HBASE-14165
Project: HBase
Issue Type: Improvement
Components: rpc
Affects Versions: 0.99.2
Reporter: cuijianwei
Assignee: cuijianwei
Priority: Minor
Fix For: 2.0.0, 1.3.0
Attachments: HBASE-14165-trunk.patch
The RWQueueRpcExecutor.queues will be initialized as:
{code}
queues = new ArrayListBlockingQueueCallRunner(writeHandlersCount +
readHandlersCount);
{code}
It seems this could be improved as:
{code}
queues = new ArrayListBlockingQueueCallRunner(numWriteQueues +
numReadQueues + numScanQueues);
{code}
Suggestions are welcomed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14259) Backport Namespace quota support to 98 branch
[
https://issues.apache.org/jira/browse/HBASE-14259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703792#comment-14703792
]
Hadoop QA commented on HBASE-14259:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12751325/HBASE-14259_v1_0.98.patch
against 0.98 branch at commit ebd34844ce9faa7ba2d97a10e00b9148bc201410.
ATTACHMENT ID: 12751325
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15172//console
This message is automatically generated.
Backport Namespace quota support to 98 branch
--
Key: HBASE-14259
URL: https://issues.apache.org/jira/browse/HBASE-14259
Project: HBase
Issue Type: New Feature
Affects Versions: 0.98.14
Reporter: Vandana Ayyalasomayajula
Assignee: Vandana Ayyalasomayajula
Priority: Minor
Attachments: HBASE-14259_v1_0.98.patch
Namespace quota support (HBASE-8410) has been backported to branch-1
(HBASE-13438). This jira would backport the same to 98 branch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14229) Flushing canceled by coprocessor still leads to memstoreSize set down
[ https://issues.apache.org/jira/browse/HBASE-14229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14229: --- Status: Patch Available (was: Open) Flushing canceled by coprocessor still leads to memstoreSize set down - Key: HBASE-14229 URL: https://issues.apache.org/jira/browse/HBASE-14229 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 1.1.1, 1.0.2, 0.98.13, 1.2.0 Reporter: sunyerui Attachments: HBASE-14229-0.98.patch, HBASE-14229-branch-1.patch A Coprocessor override public InternalScanner preFlush(final Store store, final InternalScanner scanner) and return NULL when calling this method, will cancel flush request, leaving snapshot un-flushed, and no new storefile created. But the HRegion.internalFlushCache still set down memstoreSize to 0 by totalFlushableSize. If there's no write requests anymore, the memstoreSize will remaining as 0, and no more flush quests will be processed because of the checking of memstoreSize.get() =0 at the beginning of internalFlushCache. This issue may not cause data loss, but it will confuse coprocessor users. If we argree with this, I'll apply a patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14239) Branch-1.2 AM can get stuck when meta moves
[ https://issues.apache.org/jira/browse/HBASE-14239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703887#comment-14703887 ] Enis Soztutar commented on HBASE-14239: --- [~elserj] FYI. Branch-1.2 AM can get stuck when meta moves --- Key: HBASE-14239 URL: https://issues.apache.org/jira/browse/HBASE-14239 Project: HBase Issue Type: Sub-task Components: Region Assignment Affects Versions: 1.2.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 1.2.0 Attachments: HBASE-14239-v1.patch, HBASE-14239.patch When regions are moving master can get totally stuck trying to talk to meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14239) Branch-1.2 AM can get stuck when meta moves
[ https://issues.apache.org/jira/browse/HBASE-14239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703894#comment-14703894 ] Enis Soztutar commented on HBASE-14239: --- Is this zk-less + meta is colocated in master ? Branch-1.2 AM can get stuck when meta moves --- Key: HBASE-14239 URL: https://issues.apache.org/jira/browse/HBASE-14239 Project: HBase Issue Type: Sub-task Components: Region Assignment Affects Versions: 1.2.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 1.2.0 Attachments: HBASE-14239-v1.patch, HBASE-14239.patch When regions are moving master can get totally stuck trying to talk to meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14239) Branch-1.2 AM can get stuck when meta moves
[ https://issues.apache.org/jira/browse/HBASE-14239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703894#comment-14703894 ] Enis Soztutar edited comment on HBASE-14239 at 8/19/15 11:02 PM: - Is this zk-less + meta is colocated in master ? if the meta is not in master, this should not happen I think. was (Author: enis): Is this zk-less + meta is colocated in master ? Branch-1.2 AM can get stuck when meta moves --- Key: HBASE-14239 URL: https://issues.apache.org/jira/browse/HBASE-14239 Project: HBase Issue Type: Sub-task Components: Region Assignment Affects Versions: 1.2.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 1.2.0 Attachments: HBASE-14239-v1.patch, HBASE-14239.patch When regions are moving master can get totally stuck trying to talk to meta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703900#comment-14703900
]
Nick Dimiduk commented on HBASE-14249:
--
Did you build with -Prelease ? That's the only time shaded modules are built.
{noformat}
$ patch -p1 /tmp/HBASE-14249.patch
$ mvn clean package -DskipTests -Dcheckstyle.skip -Dmaven.javadoc.skip -Prelease
...
[INFO] BUILD SUCCESS
...
$ find hbase-shaded -iname *.jar
hbase-shaded/hbase-shaded-client/target/hbase-shaded-client-1.1.2-tests.jar
hbase-shaded/hbase-shaded-client/target/hbase-shaded-client-1.1.2.jar
hbase-shaded/hbase-shaded-client/target/original-hbase-shaded-client-1.1.2.jar
hbase-shaded/hbase-shaded-server/target/hbase-shaded-server-1.1.2-tests.jar
hbase-shaded/hbase-shaded-server/target/hbase-shaded-server-1.1.2.jar
hbase-shaded/hbase-shaded-server/target/original-hbase-shaded-server-1.1.2.jar
{noformat}
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703904#comment-14703904
]
Andrew Purtell commented on HBASE-14249:
I built with 'clean install package' and looked in hbase-shaded/*/target/
Maybe with the release profile enabled I need to put those configuration
changes in each module pom. Let me try that.
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-8201) OrderedBytes: an ordered encoding strategy
[ https://issues.apache.org/jira/browse/HBASE-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703907#comment-14703907 ] Smidth Panchamia commented on HBASE-8201: - I don't know how the email was sent to this JIRA page. It is meant for DRILL-3492 :) Please ignore. OrderedBytes: an ordered encoding strategy -- Key: HBASE-8201 URL: https://issues.apache.org/jira/browse/HBASE-8201 Project: HBase Issue Type: Sub-task Components: Client Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 0.98.0, 0.95.2 Attachments: 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch Once the spec is agreed upon, it must be implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703928#comment-14703928
]
Andrew Purtell commented on HBASE-14249:
I can turn off sources jar creation with the option createSourcesJar=false to
the shade plugin, but not the tests jar.
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703933#comment-14703933
]
Andrew Purtell commented on HBASE-14249:
Attached an updated patch that will at least stop the production of source jars.
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch, HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14249:
---
Attachment: HBASE-14249.patch
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch, HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14262) Big Trunk unit tests failing with OutOfMemoryError: unable to create new native thread
stack created HBASE-14262:
-
Summary: Big Trunk unit tests failing with OutOfMemoryError:
unable to create new native thread
Key: HBASE-14262
URL: https://issues.apache.org/jira/browse/HBASE-14262
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
The bit unit tests are coming in with OOME, can't create native threads.
I was also getting the OOME locally running on MBP. git bisect got me to
HBASE-13065, where we upped the test heap for TestDistributedLogSplitting back
in feb. Around the time that this went in, we had similar OOME issues but then
it was because we were doing 32bit JVMs. It does not seem to be the case here.
A recent run failed all the below and most are OOME:
{code}
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationEndpoint
org.apache.hadoop.hbase.replication.TestPerTableCFReplication
org.apache.hadoop.hbase.wal.TestBoundedRegionGroupingProvider
org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpointNoMaster
org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.replication.TestMasterReplication
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster
org.apache.hadoop.hbase.regionserver.TestRegionFavoredNodes
org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager
org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable
org.apache.hadoop.hbase.TestGlobalMemStoreSize
org.apache.hadoop.hbase.wal.TestWALFiltering
org.apache.hadoop.hbase.replication.TestReplicationSmallTests
org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool
org.apache.hadoop.hbase.replication.TestReplicationWithTags
org.apache.hadoop.hbase.master.procedure.TestTruncateTableProcedure
org.apache.hadoop.hbase.replication.TestReplicationChangingPeerRegionservers
org.apache.hadoop.hbase.wal.TestDefaultWALProviderWithHLogKey
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.snapshot.TestMobRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure
org.apache.hadoop.hbase.wal.TestWALFactory
org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.master.procedure.TestAddColumnFamilyProcedure
org.apache.hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat
org.apache.hadoop.hbase.master.procedure.TestEnableTableProcedure
org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence
org.apache.hadoop.hbase.TestStochasticBalancerJmxMetrics
{color:red}-1 core zombie tests{color}. There are 16 zombie test(s):
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels.testVisibilityLabelsWithComplexLabels(TestVisibilityLabels.java:216)
at
org.apache.hadoop.hbase.mapred.TestTableInputFormat.testTableRecordReaderScannerFail(TestTableInputFormat.java:281)
at
org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.testMultiSlaveReplication(TestMultiSlaveReplication.java:129)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileScanning(TestHRegion.java:3799)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14243) Incorrect NOTICE file in hbase-it test-jar
[ https://issues.apache.org/jira/browse/HBASE-14243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14243: Affects Version/s: 0.98.15 Incorrect NOTICE file in hbase-it test-jar -- Key: HBASE-14243 URL: https://issues.apache.org/jira/browse/HBASE-14243 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 0.98.15, 1.0.3 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker Attachments: HBASE-14243.1.patch The hbase-it module makes use of the test resources from the hbase-server module directly. this results in a NOTICE file with placeholders instead of project attributes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14262) Big Trunk unit tests failing with OutOfMemoryError: unable to create new native thread
[
https://issues.apache.org/jira/browse/HBASE-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703993#comment-14703993
]
stack commented on HBASE-14262:
---
Above happened when we ran on:
Building remotely on H9 (Mapreduce Falcon Hadoop Pig Zookeeper Tez Hdfs) in
workspace /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build
...
/home/jenkins/jenkins-slave/tools/hudson.model.JDK/jdk-1.7u51/bin/java
java version 1.7.0_51
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
/home/jenkins/jenkins-slave/tools/hudson.model.JDK/jdk-1.7u51
java version 1.7.0_51
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
...
Here is the link:
https://builds.apache.org/job/PreCommit-HBASE-Build/15169//consoleFull
Also here on H1:
https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/15167/consoleFull
... and on H6:
https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/15164/consoleFull
Something has changed.
Big Trunk unit tests failing with OutOfMemoryError: unable to create new
native thread
Key: HBASE-14262
URL: https://issues.apache.org/jira/browse/HBASE-14262
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
The bit unit tests are coming in with OOME, can't create native threads.
I was also getting the OOME locally running on MBP. git bisect got me to
HBASE-13065, where we upped the test heap for TestDistributedLogSplitting
back in feb. Around the time that this went in, we had similar OOME issues
but then it was because we were doing 32bit JVMs. It does not seem to be the
case here.
A recent run failed all the below and most are OOME:
{code}
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationEndpoint
org.apache.hadoop.hbase.replication.TestPerTableCFReplication
org.apache.hadoop.hbase.wal.TestBoundedRegionGroupingProvider
org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpointNoMaster
org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS
org.apache.hadoop.hbase.replication.regionserver.TestRegionReplicaReplicationEndpoint
org.apache.hadoop.hbase.replication.TestMasterReplication
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster
org.apache.hadoop.hbase.regionserver.TestRegionFavoredNodes
org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
org.apache.hadoop.hbase.zookeeper.TestZKLeaderManager
org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable
org.apache.hadoop.hbase.TestGlobalMemStoreSize
org.apache.hadoop.hbase.wal.TestWALFiltering
org.apache.hadoop.hbase.replication.TestReplicationSmallTests
org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool
org.apache.hadoop.hbase.replication.TestReplicationWithTags
org.apache.hadoop.hbase.master.procedure.TestTruncateTableProcedure
org.apache.hadoop.hbase.replication.TestReplicationChangingPeerRegionservers
org.apache.hadoop.hbase.wal.TestDefaultWALProviderWithHLogKey
org.apache.hadoop.hbase.regionserver.TestPerColumnFamilyFlush
org.apache.hadoop.hbase.snapshot.TestMobRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure
org.apache.hadoop.hbase.wal.TestWALFactory
org.apache.hadoop.hbase.master.procedure.TestServerCrashProcedure
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.master.procedure.TestAddColumnFamilyProcedure
org.apache.hadoop.hbase.mapred.TestMultiTableSnapshotInputFormat
org.apache.hadoop.hbase.master.procedure.TestEnableTableProcedure
org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence
org.apache.hadoop.hbase.TestStochasticBalancerJmxMetrics
{color:red}-1 core zombie tests{color}. There are 16 zombie test(s):
[jira] [Updated] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-14249:
-
Attachment: effective-pom.xml
Effective pom with my changes to hbase-shaded/hbase-shaded-client/pom.xml from
the diff above.
{{mvn help:effective-pom -DskipTests -Dcheckstyle.skip -Dmaven.javadoc.skip
-Prelease 21 effective-pom.xml}}
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch, HBASE-14249.patch, effective-pom.xml
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14243) Incorrect NOTICE file in hbase-it test-jar
[ https://issues.apache.org/jira/browse/HBASE-14243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14243: Affects Version/s: 1.3.0 1.2.0 1.1.0 Incorrect NOTICE file in hbase-it test-jar -- Key: HBASE-14243 URL: https://issues.apache.org/jira/browse/HBASE-14243 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker Attachments: HBASE-14243.1.patch The hbase-it module makes use of the test resources from the hbase-server module directly. this results in a NOTICE file with placeholders instead of project attributes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14243) Incorrect NOTICE file in hbase-it test-jar
[ https://issues.apache.org/jira/browse/HBASE-14243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14243: Affects Version/s: (was: 1.1.0) 1.1.2 Incorrect NOTICE file in hbase-it test-jar -- Key: HBASE-14243 URL: https://issues.apache.org/jira/browse/HBASE-14243 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker Attachments: HBASE-14243.1.patch The hbase-it module makes use of the test resources from the hbase-server module directly. this results in a NOTICE file with placeholders instead of project attributes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14251) javadoc jars use LICENSE/NOTICE from primary artifact
[ https://issues.apache.org/jira/browse/HBASE-14251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14251: Status: Patch Available (was: In Progress) javadoc jars use LICENSE/NOTICE from primary artifact - Key: HBASE-14251 URL: https://issues.apache.org/jira/browse/HBASE-14251 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.2.0, 1.1.2, 0.98.15, 1.0.3 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker Attachments: HBASE-14251.1.patch Our generated javadoc jars have the same LICENSE/NOTICE files as our primary artifacts but do not include a copy of hte full source. the following modules end up with incorrect artifacts: * hbase-server * hbase-common (maybe? depends on the are-apis-copyrightable court case) * hbase-thrift -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14251) javadoc jars use LICENSE/NOTICE from primary artifact
[ https://issues.apache.org/jira/browse/HBASE-14251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14251: Attachment: HBASE-14251.1.patch -01 * place javadoc-appropriate LICENSE/NOTICE in the javadoc dir of modules that have non-standard bin/source files * use resources-plugin to filter and place them in the javadoc dir javadoc jars use LICENSE/NOTICE from primary artifact - Key: HBASE-14251 URL: https://issues.apache.org/jira/browse/HBASE-14251 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.2.0, 1.1.2, 0.98.15, 1.0.3 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker Attachments: HBASE-14251.1.patch Our generated javadoc jars have the same LICENSE/NOTICE files as our primary artifacts but do not include a copy of hte full source. the following modules end up with incorrect artifacts: * hbase-server * hbase-common (maybe? depends on the are-apis-copyrightable court case) * hbase-thrift -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14258) Make region_mover.rb script case insensitive in regard to hostname
[
https://issues.apache.org/jira/browse/HBASE-14258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703849#comment-14703849
]
Hadoop QA commented on HBASE-14258:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12751317/HBASE-14258.patch
against master branch at commit 16f8d277088987dc8d4bb1614d05ce712bfc245d.
ATTACHMENT ID: 12751317
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation, build,
or dev-support patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestRegionReplicaFailover
{color:red}-1 core zombie tests{color}. There are 9 zombie test(s):
at
org.apache.hadoop.hbase.client.TestAdmin1.testSplitAndMergeWithReplicaTable(TestAdmin1.java:1179)
at
org.apache.hadoop.hbase.client.TestFromClientSide.testIllegalTableDescriptor(TestFromClientSide.java:5550)
at
org.apache.hadoop.hbase.client.replication.TestReplicationAdminWithClusters.testEnableReplicationWhenSlaveClusterDoesntHaveTable(TestReplicationAdminWithClusters.java:65)
at
org.apache.hadoop.hbase.client.TestTableSnapshotScanner.testScanner(TestTableSnapshotScanner.java:136)
at
org.apache.hadoop.hbase.client.TestTableSnapshotScanner.testWithOfflineHBaseMultiRegion(TestTableSnapshotScanner.java:125)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15171//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15171//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15171//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15171//console
This message is automatically generated.
Make region_mover.rb script case insensitive in regard to hostname
--
Key: HBASE-14258
URL: https://issues.apache.org/jira/browse/HBASE-14258
Project: HBase
Issue Type: Bug
Reporter: Vladimir Rodionov
Assignee: Vladimir Rodionov
Priority: Minor
Fix For: 2.0.0
Attachments: HBASE-14258.patch
The script is case sensitive and fails when case of a host name being
unloaded does not match with a case of a region server name returned by HBase
API.
This doc clarifies IETF rules on case insensitivities in DNS:
https://www.ietf.org/rfc/rfc4343.txt
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-8201) OrderedBytes: an ordered encoding strategy
[ https://issues.apache.org/jira/browse/HBASE-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703853#comment-14703853 ] ASF GitHub Bot commented on HBASE-8201: --- GitHub user spanchamiamapr opened a pull request: https://github.com/apache/drill/pull/122 DRILL - 3492 - Add support for encoding of Drill data types into byte ordered format Description: This change allows encoding/decoding of data from/to 'double', 'float', 'bigint', and 'int' data types to/from OrderedBytes format. It also allows for OrderedByte encoded row-keys to be stored in ascending as well as descending order. The following JIRA added the OrderedBytes encoding to HBase: https://issues.apache.org/jira/browse/HBASE-8201 This encoding scheme will preserve the sort-order of the native data-type when it is stored as sorted byte arrays on disk. Thus, it will help the HBase storage plugin if the row-keys have been encoded in OrderedBytes format. This functionality allows us to prune the scan ranges, thus reading much lesser data from the server. Testing Done: Added a new unit-test class TestOrderedBytesConvertFunctions.java which derives from TestConvertFunctions.java class. Also added new test cases to TestHBaseFilterPushDown class that will test if we were able to push-down filters correctly and if the results are correct. You can merge this pull request into a Git repository by running: $ git pull https://github.com/spanchamiamapr/drill master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/122.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #122 commit e3642de28a5f03c702433fe581819acced7847a7 Author: spanchamia spancha...@maprtech.com Date: 2015-07-29T23:59:31Z DRILL-3364: Prune scan range if the filter is on the leading field with byte comparable encoding The change adds support to perform row-key range pruning when the row-key prefix is interpretted as UINT4_BE, TIMESTAMP_EPOCH_BE, TIME_EPOCH_BE, DATE_EPOCH_BE, UINT8_BE encoded. Testing Done: Added a unit-tests for the new feature, also ran all existing unit-tests to make sure there is no regression. commit 70e1f3b2ca0410748b9872535bb205651e86d6c9 Author: spanchamia spancha...@maprtech.com Date: 2015-07-30T05:53:04Z DRILL-3492: Add support for encoding/decoding of to/from OrderedBytes format Description: This change allows encoding/decoding of data from/to 'double', 'float', 'bigint', 'int' and 'utf8' data types to/from OrderedBytes format. It also allows for OrderedByte encoded row-keys to be stored in ascending as well as descending order. The following JIRA added the OrderedBytes encoding to HBase: https://issues.apache.org/jira/browse/HBASE-8201 This encoding scheme will preserve the sort-order of the native data-type when it is stored as sorted byte arrays on disk. Thus, it will help the HBase storage plugin if the row-keys have been encoded in OrderedBytes format. This functionality allows us to prune the scan ranges, thus reading much lesser data from the server. Testing Done: Added a new unit-test class TestOrderedBytesConvertFunctions.java which derives from TestConvertFunctions.java class. Also add new test cases to TestHBaseFilterPushDown class that will test if we were able to push-down filters correctly and if the results are correct. commit c9f8622b5cc0cf87dcdf88d73e608039556fedcb Author: Smidth Panchamia spancha...@mapr.com Date: 2015-08-19T21:51:36Z Merge remote-tracking branch 'apache/master' Conflicts: contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java contrib/storage-hbase/src/test/java/org/apache/drill/hbase/HBaseTestsSuite.java contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestTableGenerator.java commit 2daacad4ca62e753bbcad7f3637512ca810ea491 Author: Smidth Panchamia spancha...@mapr.com Date: 2015-08-19T22:18:33Z DRILL-3492 - * Remove repeated allocations of byte arrays and PositionedByteRange objects on heap(as suggested by Jason). * Remove OrderedBytes encode/decode operations on UTF8 types. Reasons - 1. These operations are slow and incur a lot of heap allocations 2. UTF8 types maintain their natural sort order when stored as binary arrays. commit 71b053006b587f39a47025302e7d3de8dcac482d Author: Smidth Panchamia spancha...@mapr.com Date: 2015-08-19T22:27:02Z DRILL-3492 - Remove test code that creates test tables with UTF8
[jira] [Updated] (HBASE-14243) Incorrect NOTICE file in hbase-it test-jar
[ https://issues.apache.org/jira/browse/HBASE-14243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14243: Affects Version/s: 1.0.3 Incorrect NOTICE file in hbase-it test-jar -- Key: HBASE-14243 URL: https://issues.apache.org/jira/browse/HBASE-14243 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 1.0.3 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker Attachments: HBASE-14243.1.patch The hbase-it module makes use of the test resources from the hbase-server module directly. this results in a NOTICE file with placeholders instead of project attributes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14260) building javadocs takes a long time
[
https://issues.apache.org/jira/browse/HBASE-14260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703868#comment-14703868
]
Andrew Purtell commented on HBASE-14260:
This is my experience also. Been this way since 0.96 / multi-module
Mavenization I think. Makes generating RCs time consuming but hasn't been
particularly painful. Thanks for filing this issue though.
building javadocs takes a long time
---
Key: HBASE-14260
URL: https://issues.apache.org/jira/browse/HBASE-14260
Project: HBase
Issue Type: Improvement
Affects Versions: 1.0.0, 2.0.0, 1.1.0, 1.2.0
Reporter: Sean Busbey
I'm not sure I have all the affected versions, but it seems that something is
amiss in making our javadocs:
{code}
mvn -Papache-release -Prelease -DskipTests clean package
... SNIP ...
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Apache HBase ... SUCCESS [ 11.149
s]
[INFO] Apache HBase - Checkstyle .. SUCCESS [ 1.249
s]
[INFO] Apache HBase - Resource Bundle . SUCCESS [ 0.539
s]
[INFO] Apache HBase - Annotations . SUCCESS [ 4.438
s]
[INFO] Apache HBase - Protocol SUCCESS [10:15
min]
[INFO] Apache HBase - Common .. SUCCESS [ 48.465
s]
[INFO] Apache HBase - Procedure ... SUCCESS [ 14.375
s]
[INFO] Apache HBase - Client .. SUCCESS [ 45.187
s]
[INFO] Apache HBase - Hadoop Compatibility SUCCESS [ 6.998
s]
[INFO] Apache HBase - Hadoop Two Compatibility SUCCESS [ 14.891
s]
[INFO] Apache HBase - Prefix Tree . SUCCESS [ 14.214
s]
[INFO] Apache HBase - Server .. SUCCESS [02:01
min]
[INFO] Apache HBase - Testing Util SUCCESS [ 12.779
s]
[INFO] Apache HBase - Thrift .. SUCCESS [01:15
min]
[INFO] Apache HBase - Shell ... SUCCESS [ 6.649
s]
[INFO] Apache HBase - Integration Tests ... SUCCESS [ 6.429
s]
[INFO] Apache HBase - Examples SUCCESS [ 13.200
s]
[INFO] Apache HBase - Rest SUCCESS [ 27.831
s]
[INFO] Apache HBase - Assembly SUCCESS [ 19.400
s]
[INFO] Apache HBase - Shaded .. SUCCESS [ 0.419
s]
[INFO] Apache HBase - Shaded - Client . SUCCESS [ 23.707
s]
[INFO] Apache HBase - Shaded - Server . SUCCESS [ 43.654
s]
[INFO] Apache HBase - Spark ... SUCCESS [02:22
min]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 21:13 min
[INFO] Finished at: 2015-08-19T15:48:00-05:00
[INFO] Final Memory: 181M/1513M
[INFO]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14261) Enhance Chaos Monkey framework by adding zookeeper and datanode fault injections.
[ https://issues.apache.org/jira/browse/HBASE-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-14261: Status: Patch Available (was: Open) Enhance Chaos Monkey framework by adding zookeeper and datanode fault injections. - Key: HBASE-14261 URL: https://issues.apache.org/jira/browse/HBASE-14261 Project: HBase Issue Type: Improvement Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Attachments: HBASE-14261-branch-1.patch One of the shortcomings of existing ChaosMonkey framework is lack of fault injections for hbase dependencies like zookeeper, hdfs etc. This patch attempts to solve this problem partially by adding datanode and zk node fault injections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14261) Enhance Chaos Monkey framework by adding zookeeper and datanode fault injections.
[ https://issues.apache.org/jira/browse/HBASE-14261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-14261: Attachment: HBASE-14261-branch-1.patch Tested the patch on 3 node cluster. Looks like fault injections are working as expected. Enhance Chaos Monkey framework by adding zookeeper and datanode fault injections. - Key: HBASE-14261 URL: https://issues.apache.org/jira/browse/HBASE-14261 Project: HBase Issue Type: Improvement Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Attachments: HBASE-14261-branch-1.patch One of the shortcomings of existing ChaosMonkey framework is lack of fault injections for hbase dependencies like zookeeper, hdfs etc. This patch attempts to solve this problem partially by adding datanode and zk node fault injections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703960#comment-14703960
]
Andrew Purtell commented on HBASE-14249:
Nope, can't specify different behavior for main and test jar.
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch, HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14250) branch-1.1 hbase-server test-jar has incorrect LICENSE
[ https://issues.apache.org/jira/browse/HBASE-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703850#comment-14703850 ] Sean Busbey commented on HBASE-14250: - When I build branch-1.1 locally, this problem doesn't show up. [~ndimiduk] can you check your local working repo for changes? branch-1.1 hbase-server test-jar has incorrect LICENSE -- Key: HBASE-14250 URL: https://issues.apache.org/jira/browse/HBASE-14250 Project: HBase Issue Type: Bug Components: build Affects Versions: 1.1.2 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker test-jar LICENSE file for hbase-server claims jquery and the orca logo are present in the jar, when they are not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14249:
---
Attachment: HBASE-14249.patch
Attached patch. No -source and -test artifacts made under the hbase-shaded
module after application.
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14249:
---
Affects Version/s: 1.3.0
Status: Patch Available (was: Open)
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-8201) OrderedBytes: an ordered encoding strategy
[ https://issues.apache.org/jira/browse/HBASE-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-8201: Assignee: Nick Dimiduk (was: Smidth Panchamia) Heya [~smidth], this issue is already implemented and closed. Perhaps you meant to apply work on some Drill JIRA? OrderedBytes: an ordered encoding strategy -- Key: HBASE-8201 URL: https://issues.apache.org/jira/browse/HBASE-8201 Project: HBase Issue Type: Sub-task Components: Client Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: 0.98.0, 0.95.2 Attachments: 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch Once the spec is agreed upon, it must be implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14249:
---
Status: Open (was: Patch Available)
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703930#comment-14703930
]
Nick Dimiduk commented on HBASE-14249:
--
Yeah, I was just trying that. Now I'm fighting with the rat plugin:
{noformat}
diff --git a/hbase-shaded/hbase-shaded-client/pom.xml
b/hbase-shaded/hbase-shaded-client/pom.xml
index c6178d6..d4d0d35 100644
--- a/hbase-shaded/hbase-shaded-client/pom.xml
+++ b/hbase-shaded/hbase-shaded-client/pom.xml
@@ -63,6 +63,22 @@
groupIdorg.apache.maven.plugins/groupId
artifactIdmaven-shade-plugin/artifactId
/plugin
+plugin
+!-- Do not make source jars --
+groupIdorg.apache.maven.plugins/groupId
+artifactIdmaven-source-plugin/artifactId
+configuration
+skipSourcetrue/skipSource
+/configuration
+/plugin
+plugin
+!-- Do not make test jars --
+groupIdorg.apache.maven.plugins/groupId
+artifactIdmaven-jar-plugin/artifactId
+configuration
+skiptrue/skip
+/configuration
+/plugin
/plugins
/build
/profile
{noformat}
{noformat}
Unapproved licenses:
/Users/ndimiduk/repos/hbase/hbase-shaded/hbase-shaded-client/target/dependency-reduced-pom.xml
/Users/ndimiduk/repos/hbase/hbase-shaded/hbase-shaded-client/target/rat.txt
{noformat}
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703953#comment-14703953
]
Andrew Purtell commented on HBASE-14249:
I don't seem to be able to stop the creation of the stub test jar, but may be
able to stuff LICENSE and NOTICE files into it. Trying
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch, HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14261) Enhance Chaos Monkey framework by adding zookeeper and datanode fault injections.
Srikanth Srungarapu created HBASE-14261: --- Summary: Enhance Chaos Monkey framework by adding zookeeper and datanode fault injections. Key: HBASE-14261 URL: https://issues.apache.org/jira/browse/HBASE-14261 Project: HBase Issue Type: Improvement Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu One of the shortcomings of existing ChaosMonkey framework is lack of fault injections for hbase dependencies like zookeeper, hdfs etc. This patch attempts to solve this problem partially by adding datanode and zk node fault injections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14222) Improve DrainBarrier
[
https://issues.apache.org/jira/browse/HBASE-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703976#comment-14703976
]
Ted Yu commented on HBASE-14222:
Patch v2 lgtm.
Improve DrainBarrier
Key: HBASE-14222
URL: https://issues.apache.org/jira/browse/HBASE-14222
Project: HBase
Issue Type: Bug
Components: util
Reporter: Hiroshi Ikeda
Assignee: Hiroshi Ikeda
Priority: Minor
Attachments: HBASE-14222-V2.patch, HBASE-14222.patch
1. {{DrainBarrier.stopAndDrainOps}} may wait forever if
{{DrainBarrier.endOp}} changes its state and calls {{Object.notifyAll}} just
before {{stopAndDrainOps}} enters the synchronized block to call
{{Object.wait}}. Moreover, {{Object.wait}} may wake up false-positively, and
{{stopAndDrainOps}} may break the block before outstanding operations are
complete.
2. Some tests for {{DrainBarrier}} catch and ignore {{AssertionError}}
explicitly thrown JUnit's {{fail}} method.
The implementation of {{DrainBarrier}} is a little complex, and I'll fix and
refactor it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703987#comment-14703987
]
Sean Busbey commented on HBASE-14249:
-
you can get a different test jar by placing things in src/test/resources htere
should be examples in e..g hbase-server.
[~ndimiduk] those target dirs should already be exempt from rat. could you do
mvn help:effective-pom (along with whatever profile activations you were
using) and attach it here?
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch, HBASE-14249.patch
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14243) Incorrect NOTICE file in hbase-it test-jar
[ https://issues.apache.org/jira/browse/HBASE-14243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14243: Resolution: Fixed Fix Version/s: 1.0.3 0.98.15 1.3.0 1.1.2 1.2.0 2.0.0 Status: Resolved (was: Patch Available) pushed to 0.98+ Incorrect NOTICE file in hbase-it test-jar -- Key: HBASE-14243 URL: https://issues.apache.org/jira/browse/HBASE-14243 Project: HBase Issue Type: Bug Components: build Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 0.98.15, 1.0.3 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker Fix For: 2.0.0, 1.2.0, 1.1.2, 1.3.0, 0.98.15, 1.0.3 Attachments: HBASE-14243.1.patch The hbase-it module makes use of the test resources from the hbase-server module directly. this results in a NOTICE file with placeholders instead of project attributes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-14249:
---
Attachment: HBASE-14249.patch
Cleaned up version of earlier patch that tells the shade plugin not to make a
source jar
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2, 1.3.0
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
Attachments: HBASE-14249.patch, HBASE-14249.patch, HBASE-14249.patch,
effective-pom.xml
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703848#comment-14703848
]
Andrew Purtell commented on HBASE-14249:
Correction, maven docs talk about maven.test.skip for both surefire and the jar
plugin, sigh. So it wasn't an incorrect thought to think it would disable test
jar creation.
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14259) Backport Namespace quota support to 98 branch
[ https://issues.apache.org/jira/browse/HBASE-14259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vandana Ayyalasomayajula updated HBASE-14259: - Attachment: HBASE-14259_v2_0.98.patch Backport Namespace quota support to 98 branch -- Key: HBASE-14259 URL: https://issues.apache.org/jira/browse/HBASE-14259 Project: HBase Issue Type: New Feature Affects Versions: 0.98.14 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: HBASE-14259_v1_0.98.patch, HBASE-14259_v2_0.98.patch Namespace quota support (HBASE-8410) has been backported to branch-1 (HBASE-13438). This jira would backport the same to 98 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14260) building javadocs takes a long time
Sean Busbey created HBASE-14260:
---
Summary: building javadocs takes a long time
Key: HBASE-14260
URL: https://issues.apache.org/jira/browse/HBASE-14260
Project: HBase
Issue Type: Improvement
Affects Versions: 1.1.0, 1.0.0, 2.0.0, 1.2.0
Reporter: Sean Busbey
I'm not sure I have all the affected versions, but it seems that something is
amiss in making our javadocs:
{code}
mvn -Papache-release -Prelease -DskipTests clean package
... SNIP ...
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Apache HBase ... SUCCESS [ 11.149 s]
[INFO] Apache HBase - Checkstyle .. SUCCESS [ 1.249 s]
[INFO] Apache HBase - Resource Bundle . SUCCESS [ 0.539 s]
[INFO] Apache HBase - Annotations . SUCCESS [ 4.438 s]
[INFO] Apache HBase - Protocol SUCCESS [10:15 min]
[INFO] Apache HBase - Common .. SUCCESS [ 48.465 s]
[INFO] Apache HBase - Procedure ... SUCCESS [ 14.375 s]
[INFO] Apache HBase - Client .. SUCCESS [ 45.187 s]
[INFO] Apache HBase - Hadoop Compatibility SUCCESS [ 6.998 s]
[INFO] Apache HBase - Hadoop Two Compatibility SUCCESS [ 14.891 s]
[INFO] Apache HBase - Prefix Tree . SUCCESS [ 14.214 s]
[INFO] Apache HBase - Server .. SUCCESS [02:01 min]
[INFO] Apache HBase - Testing Util SUCCESS [ 12.779 s]
[INFO] Apache HBase - Thrift .. SUCCESS [01:15 min]
[INFO] Apache HBase - Shell ... SUCCESS [ 6.649 s]
[INFO] Apache HBase - Integration Tests ... SUCCESS [ 6.429 s]
[INFO] Apache HBase - Examples SUCCESS [ 13.200 s]
[INFO] Apache HBase - Rest SUCCESS [ 27.831 s]
[INFO] Apache HBase - Assembly SUCCESS [ 19.400 s]
[INFO] Apache HBase - Shaded .. SUCCESS [ 0.419 s]
[INFO] Apache HBase - Shaded - Client . SUCCESS [ 23.707 s]
[INFO] Apache HBase - Shaded - Server . SUCCESS [ 43.654 s]
[INFO] Apache HBase - Spark ... SUCCESS [02:22 min]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 21:13 min
[INFO] Finished at: 2015-08-19T15:48:00-05:00
[INFO] Final Memory: 181M/1513M
[INFO]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14260) building javadocs takes a long time
[
https://issues.apache.org/jira/browse/HBASE-14260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703722#comment-14703722
]
Sean Busbey commented on HBASE-14260:
-
I'm not sure why protocol takes 10 minutes. Maybe we need to give it more
memory?
building javadocs takes a long time
---
Key: HBASE-14260
URL: https://issues.apache.org/jira/browse/HBASE-14260
Project: HBase
Issue Type: Improvement
Affects Versions: 1.0.0, 2.0.0, 1.1.0, 1.2.0
Reporter: Sean Busbey
I'm not sure I have all the affected versions, but it seems that something is
amiss in making our javadocs:
{code}
mvn -Papache-release -Prelease -DskipTests clean package
... SNIP ...
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] Apache HBase ... SUCCESS [ 11.149
s]
[INFO] Apache HBase - Checkstyle .. SUCCESS [ 1.249
s]
[INFO] Apache HBase - Resource Bundle . SUCCESS [ 0.539
s]
[INFO] Apache HBase - Annotations . SUCCESS [ 4.438
s]
[INFO] Apache HBase - Protocol SUCCESS [10:15
min]
[INFO] Apache HBase - Common .. SUCCESS [ 48.465
s]
[INFO] Apache HBase - Procedure ... SUCCESS [ 14.375
s]
[INFO] Apache HBase - Client .. SUCCESS [ 45.187
s]
[INFO] Apache HBase - Hadoop Compatibility SUCCESS [ 6.998
s]
[INFO] Apache HBase - Hadoop Two Compatibility SUCCESS [ 14.891
s]
[INFO] Apache HBase - Prefix Tree . SUCCESS [ 14.214
s]
[INFO] Apache HBase - Server .. SUCCESS [02:01
min]
[INFO] Apache HBase - Testing Util SUCCESS [ 12.779
s]
[INFO] Apache HBase - Thrift .. SUCCESS [01:15
min]
[INFO] Apache HBase - Shell ... SUCCESS [ 6.649
s]
[INFO] Apache HBase - Integration Tests ... SUCCESS [ 6.429
s]
[INFO] Apache HBase - Examples SUCCESS [ 13.200
s]
[INFO] Apache HBase - Rest SUCCESS [ 27.831
s]
[INFO] Apache HBase - Assembly SUCCESS [ 19.400
s]
[INFO] Apache HBase - Shaded .. SUCCESS [ 0.419
s]
[INFO] Apache HBase - Shaded - Client . SUCCESS [ 23.707
s]
[INFO] Apache HBase - Shaded - Server . SUCCESS [ 43.654
s]
[INFO] Apache HBase - Spark ... SUCCESS [02:22
min]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 21:13 min
[INFO] Finished at: 2015-08-19T15:48:00-05:00
[INFO] Final Memory: 181M/1513M
[INFO]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13941) Backport HBASE-13917 (Remove string comparison to identify request priority) to release branches
[ https://issues.apache.org/jira/browse/HBASE-13941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-13941: --- Resolution: Fixed Fix Version/s: (was: 1.1.3) (was: 1.0.3) 1.0.2 1.1.2 Status: Resolved (was: Patch Available) Pushed to branch-1.0 and branch-1.1. Backport HBASE-13917 (Remove string comparison to identify request priority) to release branches Key: HBASE-13941 URL: https://issues.apache.org/jira/browse/HBASE-13941 Project: HBase Issue Type: Task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 1.1.2, 1.0.2 Attachments: HBASE-13941-branch-1.0.patch Backport HBASE-13917 (Remove string comparison to identify request priority) to release branches. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14246) branch 0.98 and earlier has incorrect LICENSE/NOTICE for several dependencies
[ https://issues.apache.org/jira/browse/HBASE-14246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703830#comment-14703830 ] Andrew Purtell commented on HBASE-14246: [~busbey] did you say you had a patch for this? Happy to test / carry it forward. branch 0.98 and earlier has incorrect LICENSE/NOTICE for several dependencies - Key: HBASE-14246 URL: https://issues.apache.org/jira/browse/HBASE-14246 Project: HBase Issue Type: Bug Components: build Reporter: Sean Busbey Assignee: Sean Busbey Priority: Blocker Fix For: 0.98.14 I'll add details later, but several dependencies have incomplete information on the 0.98 and 0.94 branches. If you look at generated LICENSE/NOTICE files for unsubstituted variables (they start with $) it's a combination of ambiguous license name, and lack of copyright notice. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14249) shaded jar modules create spurious source and test jars with incorrect LICENSE/NOTICE info
[
https://issues.apache.org/jira/browse/HBASE-14249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703844#comment-14703844
]
Andrew Purtell commented on HBASE-14249:
Looking at this now.
maven.test.skip is a surefire property.
There appear config options for maven-source-plugin and maven-jar-plugin that
might help.
shaded jar modules create spurious source and test jars with incorrect
LICENSE/NOTICE info
--
Key: HBASE-14249
URL: https://issues.apache.org/jira/browse/HBASE-14249
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 2.0.0, 1.2.0, 1.1.2
Reporter: Sean Busbey
Assignee: Andrew Purtell
Priority: Blocker
the shaded jar modules don't need to create a source or test jar (because the
jars contain nothing other than META-INF)
currently we create
* the test jars are missing LICENSE
* source jars have LICENSE/NOTICE files that claim all the bundled works in
the normal jar.
{code}
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar/
hbase-shaded-server-1.1.2-sources.jar//META-INF
hbase-shaded-server-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-server-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-server-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar/
hbase-shaded-client-1.1.2-sources.jar//META-INF
hbase-shaded-client-1.1.2-sources.jar//META-INF/LICENSE
hbase-shaded-client-1.1.2-sources.jar//META-INF/MANIFEST.MF
hbase-shaded-client-1.1.2-sources.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar/
hbase-shaded-client-1.1.2-tests.jar//META-INF
hbase-shaded-client-1.1.2-tests.jar//META-INF/NOTICE
hbase-1.1.2-rc0 busbey$ find hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar/
hbase-shaded-server-1.1.2-tests.jar//META-INF
hbase-shaded-server-1.1.2-tests.jar//META-INF/NOTICE
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14233) Branch-1.2 Merging regions can result in messed up tables
[
https://issues.apache.org/jira/browse/HBASE-14233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703716#comment-14703716
]
Elliott Clark commented on HBASE-14233:
---
It's turned off by default but we have hbase.balancer.tablesOnMaster
Branch-1.2 Merging regions can result in messed up tables
-
Key: HBASE-14233
URL: https://issues.apache.org/jira/browse/HBASE-14233
Project: HBase
Issue Type: Sub-task
Components: Region Assignment
Affects Versions: 1.2.0
Reporter: Elliott Clark
Priority: Critical
Fix For: 1.2.0
{code}
15/08/17 14:10:50 INFO master.RegionStates: Transition null to
{6486b9b9409b25f10eb806ec3bad442d state=MERGING_NEW, ts=1439845850422,
server=hbase508.ash1.facebook.com,16020,1439844470302}
15/08/17 14:10:57 INFO master.AssignmentManager: Failed to record merged
region 6486b9b9409b25f10eb806ec3bad442d
15/08/17 14:10:57 ERROR master.AssignmentManager: Failed to transtion region
from {6486b9b9409b25f10eb806ec3bad442d state=MERGING_NEW, ts=1439845850422,
server=hbase508.ash1.facebook.com,16020,1439844470302} to MERGE_PONR by
hbase508.ash1.facebook.com,16020,1439844470302: Failed to record the merging
in meta
15/08/17 14:11:08 WARN master.RegionStates: THIS SHOULD NOT HAPPEN:
unexpected {6486b9b9409b25f10eb806ec3bad442d state=MERGING_NEW,
ts=1439845857580, server=hbase508.ash1.facebook.com,16020,1439844470302}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-14165) The initial size of RWQueueRpcExecutor.queues should be (numWriteQueues + numReadQueues + numScanQueues)
[
https://issues.apache.org/jira/browse/HBASE-14165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reassigned HBASE-14165:
--
Assignee: cuijianwei
The initial size of RWQueueRpcExecutor.queues should be (numWriteQueues +
numReadQueues + numScanQueues)
-
Key: HBASE-14165
URL: https://issues.apache.org/jira/browse/HBASE-14165
Project: HBase
Issue Type: Improvement
Components: rpc
Affects Versions: 0.99.2
Reporter: cuijianwei
Assignee: cuijianwei
Priority: Minor
Attachments: HBASE-14165-trunk.patch
The RWQueueRpcExecutor.queues will be initialized as:
{code}
queues = new ArrayListBlockingQueueCallRunner(writeHandlersCount +
readHandlersCount);
{code}
It seems this could be improved as:
{code}
queues = new ArrayListBlockingQueueCallRunner(numWriteQueues +
numReadQueues + numScanQueues);
{code}
Suggestions are welcomed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
