[jira] [Commented] (HBASE-11873) Hbase Version CLI enhancement
[
https://issues.apache.org/jira/browse/HBASE-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131177#comment-14131177
]
Ashish Singhi commented on HBASE-11873:
---
Thanks [~stack] for taking a look into the patch.
bq. What about the above? What happens when I build on a mac say?
This I have handled in the script like below.
{code}
which md5sum /dev/null
if [ $? != 0 ] ; then
which md5 /dev/null
if [ $? != 0 ] ; then
srcChecksum=Unknown
else
srcChecksum=`find hbase-*/src/main/ | grep -e \.java -e \.proto |
LC_ALL=C sort | xargs md5 | md5 | cut -d ' ' -f 1`
fi
else
srcChecksum=`find hbase-*/src/main/ | grep -e \.java -e \.proto |
LC_ALL=C sort | xargs md5sum | md5sum | cut -d ' ' -f 1`
fi
{code}
The first if command will check whether md5sum command exists, if it does then
it will use md5sum to generate checksum. If not, then it will check the same
for md5 command existence if it does then it will use md5 to generate the
checksum else it will mark checksum as Unknown.
I hope you understand it.
P.S: Mac uses md5 command to generate checksum
Hbase Version CLI enhancement
-
Key: HBASE-11873
URL: https://issues.apache.org/jira/browse/HBASE-11873
Project: HBase
Issue Type: Improvement
Components: build
Reporter: Guo Ruijing
Priority: Minor
Labels: beginner
Attachments: HBASE-11873-1.patch, HBASE-11873-2.patch,
HBASE-11873-3.patch, HBASE-11873.patch
Hbase Version CLI enhancements:
1) include source code checksum.
2) change Subversion to Source code repository
Existing implementation:
hadoop@localhost p4_wspaces]$ hbase version
2014-09-01 03:29:40,773 INFO [main] util.VersionInfo: HBase 0.98.1-hadoop2
2014-09-01 03:29:40,773 INFO [main] util.VersionInfo: Subversion ...
2014-09-01 03:29:40,773 INFO [main] util.VersionInfo: Compiled by ...
Expected implematation:
hadoop@localhost p4_wspaces]$ hbase version
2014-09-01 03:29:40,773 INFO [main] util.VersionInfo: HBase 0.98.1-hadoop2
2014-09-01 03:29:40,773 INFO [main] util.VersionInfo: Source code repository
...change Subversion to Source code repository
2014-09-01 03:29:40,773 INFO [main] util.VersionInfo: Compiled by ...
2014-09-01 03:29:40,773 INFO [main] util.VersionInfo: From source with
checksum eb1b9e8d63c302bed1168a7122d70 include source code checksum
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[
https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131202#comment-14131202
]
Hadoop QA commented on HBASE-11935:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12668273/11935-trunk-v2.txt
against trunk revision .
ATTACHMENT ID: 12668273
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10857//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10857//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10857//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10857//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10857//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10857//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10857//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10857//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10857//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10857//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10857//console
This message is automatically generated.
ZooKeeper connection storm after queue failover with slave cluster down
---
Key: HBASE-11935
URL: https://issues.apache.org/jira/browse/HBASE-11935
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Reporter: Lars Hofhansl
Assignee: Jesse Yates
Priority: Critical
Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1
Attachments: 11935-0.98-v2.txt, 11935-trunk-v2.txt,
hbase-11935-0.98-v0.patch, hbase-11935-0.98-v1.patch,
hbase-11935-trunk-v0.patch, hbase-11935-trunk-v1.patch,
hbase-11935-trunk-v2.patch
We just ran into a production incident with TCP SYN storms on port 2181
(zookeeper).
In our case the slave cluster was not running. When we bounced the primary
cluster we saw an unbounded number of failover threads all hammering the
hosts on the slave ZK machines (which did not run ZK at the time)... Causing
overall degradation of network performance between datacenters.
Looking at the code we noticed that the thread pool handling of the Failover
workers was probably unintended.
Patch coming soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11954) create snapshot error
[
https://issues.apache.org/jira/browse/HBASE-11954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131208#comment-14131208
]
aaron.shan commented on HBASE-11954:
I'm sorry. I will post these problems to the user list. thanks for your reply.
thank you very much!
create snapshot error
-
Key: HBASE-11954
URL: https://issues.apache.org/jira/browse/HBASE-11954
Project: HBase
Issue Type: Bug
Components: snapshots
Affects Versions: 0.98.2
Reporter: aaron.shan
When I want to create snapshot of a table, I get some exception like this:
{code}
hbase(main):004:0 snapshot 'booking', 'booking-snapshot-20140912'
ERROR: org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
ss=booking-snapshot-20140912 table=booking type=FLUSH } had an error.
Procedure booking-snapshot-20140912 {
waiting=[hbase1.data.cn,60020,1407930968832,
hbase45.data.cn,60020,1408609189376, hbase23.data.cn,60020,1407930978740,
hbase37.data.cn,60020,1408608587411, hbase46.data.cn,60020,1408609190515,
hbase6.data.cn,60020,1407930958926, hbase44.data.cn,60020,1408609188252,
hbase7.data.cn,60020,1407930960021, hbase49.data.cn,60020,1408609193897,
hbase47.data.cn,60020,1408609191647, hbase21.data.cn,60020,1407930976874,
hbase39.data.cn,60020,1408608669063, hbase13.data.cn,60020,1407930966976,
hbase15.data.cn,60020,1407930969235, hbase19.data.cn,60020,1407930973863,
hbase16.data.cn,60020,1407930971152, hbase18.data.cn,60020,1407930972762,
hbase43.data.cn,60020,1408609187126, hbase12.data.cn,60020,1407930966365,
hbase10.data.cn,60020,1407930963512, hbase3.data.cn,60020,1407930955378,
hbase11.data.cn,60020,1407930965112, hbase24.data.cn,60020,1407930979654,
hbase2.data.cn,60020,1407930954308, hbase9.data.cn,60020,1407930962354,
hbase38.data.cn,60020,1408608663894, hbase40.data.cn,60020,1408608674240,
hbase41.data.cn,60020,1408609184867, hbase4.data.cn,60020,1407930956670,
hbase36.data.cn,60020,1408608406292, hbase17.data.cn,60020,1407930972505,
hbase35.data.cn,60020,1408607982898, hbase20.data.cn,60020,1407930974993,
hbase48.data.cn,60020,1408609192763, hbase22.data.cn,60020,1407930978159,
hbase8.data.cn,60020,1407930961333] done=[] }
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:342)
at
org.apache.hadoop.hbase.master.HMaster.isSnapshotDone(HMaster.java:2905)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:40494)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at
org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:73)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by:
org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable via
timer-java.util.Timer@69db0cb4:org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
org.apache.hadoop.hbase.errorhandling.TimeoutException: Timeout elapsed!
Source:Timeout caused Foreign Exception Start:1410453067992,
End:1410453127992, diff:6, max:6 ms
at
org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:83)
at
org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:320)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:332)
... 10 more
Caused by:
org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable:
org.apache.hadoop.hbase.errorhandling.TimeoutException: Timeout elapsed!
Source:Timeout caused Foreign Exception Start:1410453067992,
End:1410453127992, diff:6, max:6 ms
at
org.apache.hadoop.hbase.errorhandling.TimeoutExceptionInjector$1.run(TimeoutExceptionInjector.java:70)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
{code}
I find the solution by google, and somebody say it maybe caused by the flush
snapshot attempting to take a region lock. See
[HBASE-7703|https://issues.apache.org/jira/browse/HBASE-7703]. But this
exception has different features.
After I flush the table, it success to create snapshot.
{code}
hbase(main):005:0 flush 'booking'
0 row(s) in 4.5220 seconds
hbase(main):006:0 snapshot 'booking', 'booking
[jira] [Commented] (HBASE-7767) Get rid of ZKTable, and table enable/disable state in ZK
[
https://issues.apache.org/jira/browse/HBASE-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131241#comment-14131241
]
Hadoop QA commented on HBASE-7767:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12668261/HBASE-7767.patch
against trunk revision .
ATTACHMENT ID: 12668261
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 45 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the trunk's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.access.TestAccessController
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
org.apache.hadoop.hbase.client.TestReplicaWithCluster
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.client.TestAdmin.testCreateTableRPCTimeOut(TestAdmin.java:1241)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10852//console
This message is automatically generated.
Get rid of ZKTable, and table enable/disable state in ZK
-
Key: HBASE-7767
URL: https://issues.apache.org/jira/browse/HBASE-7767
Project: HBase
Issue Type: Sub-task
Components: Zookeeper
Affects Versions: 2.0.0
Reporter: Enis Soztutar
Assignee: Andrey Stepachev
Attachments: HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch,
HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch,
HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch
As discussed table state in zookeeper for enable/disable state breaks our
zookeeper contract. It is also very intrusive, used from the client side,
master and region servers. We should get rid of it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11893) RowTooBigException should be in hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-11893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-11893: Status: Open (was: Patch Available) RowTooBigException should be in hbase-client module --- Key: HBASE-11893 URL: https://issues.apache.org/jira/browse/HBASE-11893 Project: HBase Issue Type: Bug Components: Client Reporter: Sean Busbey Assignee: Mikhail Antonov Priority: Minor Labels: beginner Attachments: HBASE-11893.patch, HBASE-11893.v2.patch RowTooBigException is Public get thrown from client calls. It should be in hbase-client instead of hbase-server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11893) RowTooBigException should be in hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-11893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-11893: Attachment: HBASE-11893.v2.patch updated patch RowTooBigException should be in hbase-client module --- Key: HBASE-11893 URL: https://issues.apache.org/jira/browse/HBASE-11893 Project: HBase Issue Type: Bug Components: Client Reporter: Sean Busbey Assignee: Mikhail Antonov Priority: Minor Labels: beginner Attachments: HBASE-11893.patch, HBASE-11893.v2.patch RowTooBigException is Public get thrown from client calls. It should be in hbase-client instead of hbase-server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11893) RowTooBigException should be in hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-11893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-11893: Status: Patch Available (was: Open) RowTooBigException should be in hbase-client module --- Key: HBASE-11893 URL: https://issues.apache.org/jira/browse/HBASE-11893 Project: HBase Issue Type: Bug Components: Client Reporter: Sean Busbey Assignee: Mikhail Antonov Priority: Minor Labels: beginner Attachments: HBASE-11893.patch, HBASE-11893.v2.patch RowTooBigException is Public get thrown from client calls. It should be in hbase-client instead of hbase-server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-5974) Scanner retry behavior with RPC timeout on next() seems incorrect
[ https://issues.apache.org/jira/browse/HBASE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131264#comment-14131264 ] Liu Shaohui commented on HBASE-5974: [~lhofhansl] This is a critical bug in scan in 0.94. We should fix it in 0.94. What's your suggestion? Thanks Scanner retry behavior with RPC timeout on next() seems incorrect - Key: HBASE-5974 URL: https://issues.apache.org/jira/browse/HBASE-5974 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.95.2 Reporter: Todd Lipcon Assignee: Anoop Sam John Priority: Critical Fix For: 0.95.0 Attachments: 5974_94-V4.patch, 5974_trunk-V2.patch, 5974_trunk-V3.patch, 5974_trunk.patch, HBASE-5974_0.94.patch, HBASE-5974_94-V2.patch, HBASE-5974_94-V3.patch I'm seeing the following behavior: - set RPC timeout to a short value - call next() for some batch of rows, big enough so the client times out before the result is returned - the HConnectionManager stuff will retry the next() call to the same server. At this point, one of two things can happen: 1) the previous next() call will still be processing, in which case you get a LeaseException, because it was removed from the map during the processing, or 2) the next() call will succeed but skip the prior batch of rows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11646) Handle the MOB in compaction
[ https://issues.apache.org/jira/browse/HBASE-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-11646: - Attachment: HBASE-11646-V6.diff Upload the latest patch (V6). Refine the code and comments, add methods to parse the value of mob ref cell. Handle the MOB in compaction Key: HBASE-11646 URL: https://issues.apache.org/jira/browse/HBASE-11646 Project: HBase Issue Type: Sub-task Components: Compaction Affects Versions: hbase-11339 Reporter: Jingcheng Du Assignee: Jingcheng Du Fix For: hbase-11339 Attachments: HBASE-11646-V2.diff, HBASE-11646-V3.diff, HBASE-11646-V4.diff, HBASE-11646-V5.diff, HBASE-11646-V6.diff, HBASE-11646.diff In the updated MOB design however, admins can set CF level thresholds that would force cell values the threshold to use the MOB write path instead of the traditional path. There are two cases where mobs need to interact with this threshold 1) How do we handle the case when the threshold size is changed? 2) Today, you can bulkload hfiles that contain MOBs. These cells will work as normal inside hbase. Unfortunately the cells with MOBs in them will never benefit form the MOB write path. The proposal here is to modify compaction in mob enabled cf's such that the threshold value is honored with compactions. This handles case #1 -- elements that should be moved out of the normal hfiles get 'compacted' into refs and mob hfiles, and values that should be pulled into the cf get derefed and written out wholy in the compaction. For case #2, we can maintain the same behavior and compaction would move data into the mob writepath/lifecycle. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11646) Handle the MOB in compaction
[
https://issues.apache.org/jira/browse/HBASE-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131370#comment-14131370
]
Hadoop QA commented on HBASE-11646:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12668311/HBASE-11646-V6.diff
against trunk revision .
ATTACHMENT ID: 12668311
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10859//console
This message is automatically generated.
Handle the MOB in compaction
Key: HBASE-11646
URL: https://issues.apache.org/jira/browse/HBASE-11646
Project: HBase
Issue Type: Sub-task
Components: Compaction
Affects Versions: hbase-11339
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Fix For: hbase-11339
Attachments: HBASE-11646-V2.diff, HBASE-11646-V3.diff,
HBASE-11646-V4.diff, HBASE-11646-V5.diff, HBASE-11646-V6.diff,
HBASE-11646.diff
In the updated MOB design however, admins can set CF level thresholds that
would force cell values the threshold to use the MOB write path instead of
the traditional path. There are two cases where mobs need to interact with
this threshold
1) How do we handle the case when the threshold size is changed?
2) Today, you can bulkload hfiles that contain MOBs. These cells will work
as normal inside hbase. Unfortunately the cells with MOBs in them will never
benefit form the MOB write path.
The proposal here is to modify compaction in mob enabled cf's such that the
threshold value is honored with compactions. This handles case #1 --
elements that should be moved out of the normal hfiles get 'compacted' into
refs and mob hfiles, and values that should be pulled into the cf get derefed
and written out wholy in the compaction. For case #2, we can maintain the
same behavior and compaction would move data into the mob writepath/lifecycle.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-11957) Backport HBASE-5974 to 0.94
Liu Shaohui created HBASE-11957: --- Summary: Backport HBASE-5974 to 0.94 Key: HBASE-11957 URL: https://issues.apache.org/jira/browse/HBASE-11957 Project: HBase Issue Type: Bug Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Critical Fix For: 0.94.24 HBASE-5974:Scanner retry behavior with RPC timeout on next() seems incorrect, which cause data missing in hbase scan. I think we should fix it in 0.94. [~lhofhansl] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11957) Backport HBASE-5974 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-11957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Shaohui updated HBASE-11957: Attachment: HBASE-5974-0.94-v1.diff Rebase the patch v3 in HBASE-5974 for 0.94 Backport HBASE-5974 to 0.94 --- Key: HBASE-11957 URL: https://issues.apache.org/jira/browse/HBASE-11957 Project: HBase Issue Type: Bug Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Critical Fix For: 0.94.24 Attachments: HBASE-5974-0.94-v1.diff HBASE-5974:Scanner retry behavior with RPC timeout on next() seems incorrect, which cause data missing in hbase scan. I think we should fix it in 0.94. [~lhofhansl] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11893) RowTooBigException should be in hbase-client module
[
https://issues.apache.org/jira/browse/HBASE-11893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131428#comment-14131428
]
Hadoop QA commented on HBASE-11893:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12668297/HBASE-11893.v2.patch
against trunk revision .
ATTACHMENT ID: 12668297
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan2
org.apache.hadoop.hbase.master.TestMasterFailover
org.apache.hadoop.hbase.regionserver.TestRowTooBig
org.apache.hadoop.hbase.client.TestReplicaWithCluster
org.apache.hadoop.hbase.replication.TestPerTableCFReplication
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10858//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10858//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10858//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10858//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10858//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10858//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10858//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10858//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10858//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10858//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10858//console
This message is automatically generated.
RowTooBigException should be in hbase-client module
---
Key: HBASE-11893
URL: https://issues.apache.org/jira/browse/HBASE-11893
Project: HBase
Issue Type: Bug
Components: Client
Reporter: Sean Busbey
Assignee: Mikhail Antonov
Priority: Minor
Labels: beginner
Attachments: HBASE-11893.patch, HBASE-11893.v2.patch
RowTooBigException is Public get thrown from client calls. It should be in
hbase-client instead of hbase-server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11957) Backport HBASE-5974 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-11957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131432#comment-14131432 ] Anoop Sam John commented on HBASE-11957: Haven't gone through the back porting patch. One quick question. Do we make sure client to server compatibility? ie. older version client can talk with new server(0.94.24 with this fix) and new client to old server Backport HBASE-5974 to 0.94 --- Key: HBASE-11957 URL: https://issues.apache.org/jira/browse/HBASE-11957 Project: HBase Issue Type: Bug Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Critical Fix For: 0.94.24 Attachments: HBASE-5974-0.94-v1.diff HBASE-5974:Scanner retry behavior with RPC timeout on next() seems incorrect, which cause data missing in hbase scan. I think we should fix it in 0.94. [~lhofhansl] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11957) Backport HBASE-5974 to 0.94
[
https://issues.apache.org/jira/browse/HBASE-11957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131447#comment-14131447
]
Liu Shaohui commented on HBASE-11957:
-
[~anoop.hbase]
{code}
Haven't gone through the back porting patch. One quick question. Do we make
sure client to server compatibility? ie. older version client can talk with new
server(0.94.24 with this fix) and new client to old server
{code}
Yes. Older version client can talk with new server with old next api.
New client try to use the new next api first. If there is no such methond, it
will switch to use old api.
See the code in ScannerCallable#call
Backport HBASE-5974 to 0.94
---
Key: HBASE-11957
URL: https://issues.apache.org/jira/browse/HBASE-11957
Project: HBase
Issue Type: Bug
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Critical
Fix For: 0.94.24
Attachments: HBASE-5974-0.94-v1.diff
HBASE-5974:Scanner retry behavior with RPC timeout on next() seems incorrect,
which cause data missing in hbase scan.
I think we should fix it in 0.94.
[~lhofhansl]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-11958) Add documents about snapshot owner
Liu Shaohui created HBASE-11958: --- Summary: Add documents about snapshot owner Key: HBASE-11958 URL: https://issues.apache.org/jira/browse/HBASE-11958 Project: HBase Issue Type: Sub-task Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor HBASE-11869 introduct snapshot owner feature. We need to add documents about it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11958) Add documents about snapshot owner
[ https://issues.apache.org/jira/browse/HBASE-11958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Shaohui updated HBASE-11958: Description: HBASE-11869 introduces snapshot owner feature. We need to add documents about it. (was: HBASE-11869 introduct snapshot owner feature. We need to add documents about it.) Add documents about snapshot owner -- Key: HBASE-11958 URL: https://issues.apache.org/jira/browse/HBASE-11958 Project: HBase Issue Type: Sub-task Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0 HBASE-11869 introduces snapshot owner feature. We need to add documents about it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11897) Add append and remove peer table-cfs cmds for replication
[ https://issues.apache.org/jira/browse/HBASE-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131480#comment-14131480 ] Liu Shaohui commented on HBASE-11897: - [~enis] Could you help to push this issue? Thanks. Add append and remove peer table-cfs cmds for replication - Key: HBASE-11897 URL: https://issues.apache.org/jira/browse/HBASE-11897 Project: HBase Issue Type: Improvement Affects Versions: 2.0.0 Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11897-trunk-v1.diff, HBASE-11897-trunk-v2.diff, hbase-11897_v3.patch HBASE-8751 introduces the tables/table-column families config for a replication peer. It's very flexible for practical replication in hbase clusters. But it is easy to make mistakes during add or remove a table/table-column family for a existing peer, especially when the table-cfs is very long, for we need to copy the current table-cfs of the peer first, and then add or remove a table/table-column family to/from the table-cfs, at last set back the table-cfs using the cmd: set_peer_tableCFs. So we implements two new cmds: append_peer_tableCFs and remove_peer_tableCFs to do the operation of adding and removing a table/table-column family. They are useful operation tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11897) Add append and remove peer table-cfs cmds for replication
[ https://issues.apache.org/jira/browse/HBASE-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131486#comment-14131486 ] Liu Shaohui commented on HBASE-11897: - The failed test: TestClassFinder has no relations with this patch. And the test passed in local test. Add append and remove peer table-cfs cmds for replication - Key: HBASE-11897 URL: https://issues.apache.org/jira/browse/HBASE-11897 Project: HBase Issue Type: Improvement Affects Versions: 2.0.0 Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11897-trunk-v1.diff, HBASE-11897-trunk-v2.diff, hbase-11897_v3.patch HBASE-8751 introduces the tables/table-column families config for a replication peer. It's very flexible for practical replication in hbase clusters. But it is easy to make mistakes during add or remove a table/table-column family for a existing peer, especially when the table-cfs is very long, for we need to copy the current table-cfs of the peer first, and then add or remove a table/table-column family to/from the table-cfs, at last set back the table-cfs using the cmd: set_peer_tableCFs. So we implements two new cmds: append_peer_tableCFs and remove_peer_tableCFs to do the operation of adding and removing a table/table-column family. They are useful operation tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11893) RowTooBigException should be in hbase-client module
[
https://issues.apache.org/jira/browse/HBASE-11893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131560#comment-14131560
]
Sean Busbey commented on HBASE-11893:
-
{quote}
diff --git
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RowTooBigException.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RowTooBigException.java
index 56d3872..60f3293 100644
---
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RowTooBigException.java
+++
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RowTooBigException.java
@@ -25,8 +25,11 @@ import org.apache.hadoop.hbase.RegionException;
* Gets or Scans throw this exception if running without in-row scan flag
* set and row size appears to exceed max configured size (configurable via
* hbase.table.max.rowsize).
+ *
+ * @deprecated use {@link org.apache.hadoop.hbase.client.RowTooBigException}
instead.
*/
@InterfaceAudience.Public
+@Deprecated
public class RowTooBigException extends RegionException {
{quote}
regionserver.RowTooBigException should extend client.RowTooBigException. That
way the server can throw it and clients that are expecting either the old or
the client version will work.
After making this change TestRowTooBig should stop failing. AFAICT the other
failures from QA are unrelated, they pass locally.
RowTooBigException should be in hbase-client module
---
Key: HBASE-11893
URL: https://issues.apache.org/jira/browse/HBASE-11893
Project: HBase
Issue Type: Bug
Components: Client
Reporter: Sean Busbey
Assignee: Mikhail Antonov
Priority: Minor
Labels: beginner
Attachments: HBASE-11893.patch, HBASE-11893.v2.patch
RowTooBigException is Public get thrown from client calls. It should be in
hbase-client instead of hbase-server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11879) Change TableInputFormatBase to take interface arguments
[
https://issues.apache.org/jira/browse/HBASE-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Solomon Duskis updated HBASE-11879:
---
Attachment: HBASE_11825.patch
Most of the work in this patch was from Enis's patch in HBASE-10602. There
were a few things that changed since April, so I did my best to make sure that
differences between Enis's patch and the current master are resolved. Given
that this is my first patch, I'm pretty sure that I've missed something, but
not sure what.
Change TableInputFormatBase to take interface arguments
---
Key: HBASE-11879
URL: https://issues.apache.org/jira/browse/HBASE-11879
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Carter
Attachments: HBASE_11825.patch
As part of the ongoing interface abstraction work, I'm now investigating
{{TableInputFormatBase}}, which has two methods that break encapsulation:
{code}
protected HTable getHTable();
protected void setHTable(HTable table);
{code}
While these are protected methods, the base @InterfaceAudience.Public is
abstract, meaning that it supports extension by user code.
I propose deprecating these two methods and replacing them with these four,
once the Table interface is merged:
{code}
protected Table getTable();
protected void setTable(Table table);
protected RegionLocator getRegionLocator();
protected void setRegionLocator(RegionLocator regionLocator);
{code}
Since users will frequently call {{setTable}} and {{setRegionLocator}}
together, it probably also makes sense to add the following convenience
method:
{code}
protected void setTableAndRegionLocator(Table table, RegionLocator
regionLocator);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11879) Change TableInputFormatBase to take interface arguments
[
https://issues.apache.org/jira/browse/HBASE-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Solomon Duskis updated HBASE-11879:
---
Attachment: (was: HBASE_11825.patch)
Change TableInputFormatBase to take interface arguments
---
Key: HBASE-11879
URL: https://issues.apache.org/jira/browse/HBASE-11879
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Carter
As part of the ongoing interface abstraction work, I'm now investigating
{{TableInputFormatBase}}, which has two methods that break encapsulation:
{code}
protected HTable getHTable();
protected void setHTable(HTable table);
{code}
While these are protected methods, the base @InterfaceAudience.Public is
abstract, meaning that it supports extension by user code.
I propose deprecating these two methods and replacing them with these four,
once the Table interface is merged:
{code}
protected Table getTable();
protected void setTable(Table table);
protected RegionLocator getRegionLocator();
protected void setRegionLocator(RegionLocator regionLocator);
{code}
Since users will frequently call {{setTable}} and {{setRegionLocator}}
together, it probably also makes sense to add the following convenience
method:
{code}
protected void setTableAndRegionLocator(Table table, RegionLocator
regionLocator);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11879) Change TableInputFormatBase to take interface arguments
[
https://issues.apache.org/jira/browse/HBASE-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131643#comment-14131643
]
Solomon Duskis commented on HBASE-11879:
Ugh. My first comment was on the wrong issue. It was meant for HBASE-11825.
Change TableInputFormatBase to take interface arguments
---
Key: HBASE-11879
URL: https://issues.apache.org/jira/browse/HBASE-11879
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Carter
As part of the ongoing interface abstraction work, I'm now investigating
{{TableInputFormatBase}}, which has two methods that break encapsulation:
{code}
protected HTable getHTable();
protected void setHTable(HTable table);
{code}
While these are protected methods, the base @InterfaceAudience.Public is
abstract, meaning that it supports extension by user code.
I propose deprecating these two methods and replacing them with these four,
once the Table interface is merged:
{code}
protected Table getTable();
protected void setTable(Table table);
protected RegionLocator getRegionLocator();
protected void setRegionLocator(RegionLocator regionLocator);
{code}
Since users will frequently call {{setTable}} and {{setRegionLocator}}
together, it probably also makes sense to add the following convenience
method:
{code}
protected void setTableAndRegionLocator(Table table, RegionLocator
regionLocator);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11825) Create Connection and ConnectionManager
[
https://issues.apache.org/jira/browse/HBASE-11825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Solomon Duskis updated HBASE-11825:
---
Attachment: HBASE_11825.patch
Most of the work in this patch was from [~enis]'s patch in HBASE-10602. There
were a few things that changed since April, so I did my best to make sure that
differences between [~enis]'s patch and the current master are resolved. Given
that this is my first patch, I'm pretty sure that I've missed something, but
not sure what.
Create Connection and ConnectionManager
---
Key: HBASE-11825
URL: https://issues.apache.org/jira/browse/HBASE-11825
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Solomon Duskis
Priority: Critical
Fix For: 2.0.0, 0.99.1
Attachments: HBASE_11825.patch
This is further cleanup of the HBase interface for 1.0 after implementing the
new Table and Admin interfaces. Following Enis's guidelines in HBASE-10602,
this JIRA will generate a new ConnectionManager to replace HCM and Connection
to replace HConnection.
For more detail, this JIRA intends to implement this portion:
{code}
interface Connection extends Closeable{
Table getTable(), and rest of HConnection methods
getAdmin()
// no deprecated methods (cache related etc)
}
@Deprecated
interface HConnection extends Connection {
@Deprecated
HTableInterface getTable()
// users are encouraged to use Connection
}
class ConnectionManager {
createConnection(Configuration) // not sure whether we want a static
factory method to create connections or a ctor
}
@Deprecated
class HCM extends ConnectionManager {
// users are encouraged to use ConnectionManager
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-7767) Get rid of ZKTable, and table enable/disable state in ZK
[ https://issues.apache.org/jira/browse/HBASE-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-7767: Status: Open (was: Patch Available) Get rid of ZKTable, and table enable/disable state in ZK - Key: HBASE-7767 URL: https://issues.apache.org/jira/browse/HBASE-7767 Project: HBase Issue Type: Sub-task Components: Zookeeper Affects Versions: 2.0.0 Reporter: Enis Soztutar Assignee: Andrey Stepachev Attachments: HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch As discussed table state in zookeeper for enable/disable state breaks our zookeeper contract. It is also very intrusive, used from the client side, master and region servers. We should get rid of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-7767) Get rid of ZKTable, and table enable/disable state in ZK
[ https://issues.apache.org/jira/browse/HBASE-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-7767: Status: Patch Available (was: Open) Get rid of ZKTable, and table enable/disable state in ZK - Key: HBASE-7767 URL: https://issues.apache.org/jira/browse/HBASE-7767 Project: HBase Issue Type: Sub-task Components: Zookeeper Affects Versions: 2.0.0 Reporter: Enis Soztutar Assignee: Andrey Stepachev Attachments: HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch As discussed table state in zookeeper for enable/disable state breaks our zookeeper contract. It is also very intrusive, used from the client side, master and region servers. We should get rid of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-7767) Get rid of ZKTable, and table enable/disable state in ZK
[ https://issues.apache.org/jira/browse/HBASE-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-7767: Attachment: HBASE-7767.patch Get rid of ZKTable, and table enable/disable state in ZK - Key: HBASE-7767 URL: https://issues.apache.org/jira/browse/HBASE-7767 Project: HBase Issue Type: Sub-task Components: Zookeeper Affects Versions: 2.0.0 Reporter: Enis Soztutar Assignee: Andrey Stepachev Attachments: HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch As discussed table state in zookeeper for enable/disable state breaks our zookeeper contract. It is also very intrusive, used from the client side, master and region servers. We should get rid of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-7767) Get rid of ZKTable, and table enable/disable state in ZK
[ https://issues.apache.org/jira/browse/HBASE-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-7767: Status: Open (was: Patch Available) Get rid of ZKTable, and table enable/disable state in ZK - Key: HBASE-7767 URL: https://issues.apache.org/jira/browse/HBASE-7767 Project: HBase Issue Type: Sub-task Components: Zookeeper Affects Versions: 2.0.0 Reporter: Enis Soztutar Assignee: Andrey Stepachev Attachments: HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch As discussed table state in zookeeper for enable/disable state breaks our zookeeper contract. It is also very intrusive, used from the client side, master and region servers. We should get rid of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-7767) Get rid of ZKTable, and table enable/disable state in ZK
[ https://issues.apache.org/jira/browse/HBASE-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-7767: Status: Patch Available (was: Open) Get rid of ZKTable, and table enable/disable state in ZK - Key: HBASE-7767 URL: https://issues.apache.org/jira/browse/HBASE-7767 Project: HBase Issue Type: Sub-task Components: Zookeeper Affects Versions: 2.0.0 Reporter: Enis Soztutar Assignee: Andrey Stepachev Attachments: HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch As discussed table state in zookeeper for enable/disable state breaks our zookeeper contract. It is also very intrusive, used from the client side, master and region servers. We should get rid of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11879) Change TableInputFormatBase to take interface arguments
[
https://issues.apache.org/jira/browse/HBASE-11879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131668#comment-14131668
]
Solomon Duskis commented on HBASE-11879:
Does approach #2 also need to have a TableName passed in order to get both the
Table and RegionLocator?
Change TableInputFormatBase to take interface arguments
---
Key: HBASE-11879
URL: https://issues.apache.org/jira/browse/HBASE-11879
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Carter
As part of the ongoing interface abstraction work, I'm now investigating
{{TableInputFormatBase}}, which has two methods that break encapsulation:
{code}
protected HTable getHTable();
protected void setHTable(HTable table);
{code}
While these are protected methods, the base @InterfaceAudience.Public is
abstract, meaning that it supports extension by user code.
I propose deprecating these two methods and replacing them with these four,
once the Table interface is merged:
{code}
protected Table getTable();
protected void setTable(Table table);
protected RegionLocator getRegionLocator();
protected void setRegionLocator(RegionLocator regionLocator);
{code}
Since users will frequently call {{setTable}} and {{setRegionLocator}}
together, it probably also makes sense to add the following convenience
method:
{code}
protected void setTableAndRegionLocator(Table table, RegionLocator
regionLocator);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11825) Create Connection and ConnectionManager
[
https://issues.apache.org/jira/browse/HBASE-11825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131671#comment-14131671
]
Solomon Duskis commented on HBASE-11825:
I had to revert some of the Admin - HBaseAdmin changes after I changed Admin's
getConnection method to return a Connection instead of an HConnection.
Admin.getConnection() returns a Connection and HBaseAdmin.getConnection()
returns an HConnection. Any place that needed HConnection's internal methods
now use HBaseAdmin again.
Create Connection and ConnectionManager
---
Key: HBASE-11825
URL: https://issues.apache.org/jira/browse/HBASE-11825
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Solomon Duskis
Priority: Critical
Fix For: 2.0.0, 0.99.1
Attachments: HBASE_11825.patch
This is further cleanup of the HBase interface for 1.0 after implementing the
new Table and Admin interfaces. Following Enis's guidelines in HBASE-10602,
this JIRA will generate a new ConnectionManager to replace HCM and Connection
to replace HConnection.
For more detail, this JIRA intends to implement this portion:
{code}
interface Connection extends Closeable{
Table getTable(), and rest of HConnection methods
getAdmin()
// no deprecated methods (cache related etc)
}
@Deprecated
interface HConnection extends Connection {
@Deprecated
HTableInterface getTable()
// users are encouraged to use Connection
}
class ConnectionManager {
createConnection(Configuration) // not sure whether we want a static
factory method to create connections or a ctor
}
@Deprecated
class HCM extends ConnectionManager {
// users are encouraged to use ConnectionManager
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-11959) TestAssignmentManagerOnCluster is falky
Jimmy Xiang created HBASE-11959: --- Summary: TestAssignmentManagerOnCluster is falky Key: HBASE-11959 URL: https://issues.apache.org/jira/browse/HBASE-11959 Project: HBase Issue Type: Bug Affects Versions: 0.99.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang https://builds.apache.org/job/HBase-1.0/178/testReport/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11959) TestAssignmentManagerOnCluster is flaky
[ https://issues.apache.org/jira/browse/HBASE-11959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11959: Summary: TestAssignmentManagerOnCluster is flaky (was: TestAssignmentManagerOnCluster is falky) TestAssignmentManagerOnCluster is flaky --- Key: HBASE-11959 URL: https://issues.apache.org/jira/browse/HBASE-11959 Project: HBase Issue Type: Bug Affects Versions: 0.99.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang https://builds.apache.org/job/HBase-1.0/178/testReport/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11959) TestAssignmentManagerOnCluster is flaky
[ https://issues.apache.org/jira/browse/HBASE-11959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131687#comment-14131687 ] Jimmy Xiang commented on HBASE-11959: - This is very likely related to HBASE-11604. After which, we don't put meta on master any more by default. Some tests will be flaky if meta region server is restarted in the middle. TestAssignmentManagerOnCluster is flaky --- Key: HBASE-11959 URL: https://issues.apache.org/jira/browse/HBASE-11959 Project: HBase Issue Type: Bug Affects Versions: 0.99.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang https://builds.apache.org/job/HBase-1.0/178/testReport/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11959) TestAssignmentManagerOnCluster is flaky
[ https://issues.apache.org/jira/browse/HBASE-11959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131690#comment-14131690 ] stack commented on HBASE-11959: --- Makes sense. TestAssignmentManagerOnCluster is flaky --- Key: HBASE-11959 URL: https://issues.apache.org/jira/browse/HBASE-11959 Project: HBase Issue Type: Bug Affects Versions: 0.99.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang https://builds.apache.org/job/HBase-1.0/178/testReport/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-11960) Provide a sample to show how to use Thrift client authentication
Jimmy Xiang created HBASE-11960: --- Summary: Provide a sample to show how to use Thrift client authentication Key: HBASE-11960 URL: https://issues.apache.org/jira/browse/HBASE-11960 Project: HBase Issue Type: Task Reporter: Jimmy Xiang Assignee: Jimmy Xiang Now Thrift server can authenticate clients. However, many people asked how to use it. Although we have some info in the refguide, it doesn't mention how to change the client to turn it on. A sample should help. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-11961) Document region state transitions
Jimmy Xiang created HBASE-11961: --- Summary: Document region state transitions Key: HBASE-11961 URL: https://issues.apache.org/jira/browse/HBASE-11961 Project: HBase Issue Type: Task Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Document the region state transitions in the refguide. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11958) Add documents about snapshot owner
[ https://issues.apache.org/jira/browse/HBASE-11958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11958: -- Component/s: documentation Add documents about snapshot owner -- Key: HBASE-11958 URL: https://issues.apache.org/jira/browse/HBASE-11958 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0 HBASE-11869 introduces snapshot owner feature. We need to add documents about it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11897) Add append and remove peer table-cfs cmds for replication
[ https://issues.apache.org/jira/browse/HBASE-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11897: -- Resolution: Fixed Fix Version/s: (was: 0.98.7) Release Note: Adds two new shell commands for operators to add and remove peers at the columnfamily scope. Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the patch [~liushaohui]. Did not apply clean to 0.98 so skipped on backporting. Addressed javadoc warning before commit. Add append and remove peer table-cfs cmds for replication - Key: HBASE-11897 URL: https://issues.apache.org/jira/browse/HBASE-11897 Project: HBase Issue Type: Improvement Components: Operability Affects Versions: 2.0.0 Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0, 0.99.1 Attachments: HBASE-11897-trunk-v1.diff, HBASE-11897-trunk-v2.diff, hbase-11897_v3.patch HBASE-8751 introduces the tables/table-column families config for a replication peer. It's very flexible for practical replication in hbase clusters. But it is easy to make mistakes during add or remove a table/table-column family for a existing peer, especially when the table-cfs is very long, for we need to copy the current table-cfs of the peer first, and then add or remove a table/table-column family to/from the table-cfs, at last set back the table-cfs using the cmd: set_peer_tableCFs. So we implements two new cmds: append_peer_tableCFs and remove_peer_tableCFs to do the operation of adding and removing a table/table-column family. They are useful operation tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11897) Add append and remove peer table-cfs cmds for replication
[ https://issues.apache.org/jira/browse/HBASE-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11897: -- Component/s: Operability Add append and remove peer table-cfs cmds for replication - Key: HBASE-11897 URL: https://issues.apache.org/jira/browse/HBASE-11897 Project: HBase Issue Type: Improvement Components: Operability Affects Versions: 2.0.0 Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0, 0.99.1 Attachments: HBASE-11897-trunk-v1.diff, HBASE-11897-trunk-v2.diff, hbase-11897_v3.patch HBASE-8751 introduces the tables/table-column families config for a replication peer. It's very flexible for practical replication in hbase clusters. But it is easy to make mistakes during add or remove a table/table-column family for a existing peer, especially when the table-cfs is very long, for we need to copy the current table-cfs of the peer first, and then add or remove a table/table-column family to/from the table-cfs, at last set back the table-cfs using the cmd: set_peer_tableCFs. So we implements two new cmds: append_peer_tableCFs and remove_peer_tableCFs to do the operation of adding and removing a table/table-column family. They are useful operation tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11959) TestAssignmentManagerOnCluster is flaky
[ https://issues.apache.org/jira/browse/HBASE-11959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-11959: Attachment: hbase-11959.patch This is for branch 1 only. This patch puts meta on master during this test. We have separate tests to test meta region server recovery, where meta is moved to a region server from master at first. TestAssignmentManagerOnCluster is flaky --- Key: HBASE-11959 URL: https://issues.apache.org/jira/browse/HBASE-11959 Project: HBase Issue Type: Bug Affects Versions: 0.99.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Attachments: hbase-11959.patch https://builds.apache.org/job/HBase-1.0/178/testReport/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-11962) Port HBASE-11897 Add append and remove peer table-cfs cmds for replication to 0.98
Ted Yu created HBASE-11962: -- Summary: Port HBASE-11897 Add append and remove peer table-cfs cmds for replication to 0.98 Key: HBASE-11962 URL: https://issues.apache.org/jira/browse/HBASE-11962 Project: HBase Issue Type: Improvement Reporter: Ted Yu Priority: Minor Fix For: 0.98.7 This issue is to backport the commands for appending and removing peer table-cfs for replication to 0.98 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11962) Port HBASE-11897 Add append and remove peer table-cfs cmds for replication to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-11962: --- Description: This issue is to backport the commands for appending and removing peer table-cfs for replication to 0.98 Two new commands, append_peer_tableCFs and remove_peer_tableCFs, are added to do the operation of adding and removing a table/table-column family. was:This issue is to backport the commands for appending and removing peer table-cfs for replication to 0.98 Port HBASE-11897 Add append and remove peer table-cfs cmds for replication to 0.98 Key: HBASE-11962 URL: https://issues.apache.org/jira/browse/HBASE-11962 Project: HBase Issue Type: Improvement Reporter: Ted Yu Priority: Minor Fix For: 0.98.7 This issue is to backport the commands for appending and removing peer table-cfs for replication to 0.98 Two new commands, append_peer_tableCFs and remove_peer_tableCFs, are added to do the operation of adding and removing a table/table-column family. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11897) Add append and remove peer table-cfs cmds for replication
[ https://issues.apache.org/jira/browse/HBASE-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131753#comment-14131753 ] Ted Yu commented on HBASE-11897: Logged HBASE-11962 for the backport Add append and remove peer table-cfs cmds for replication - Key: HBASE-11897 URL: https://issues.apache.org/jira/browse/HBASE-11897 Project: HBase Issue Type: Improvement Components: Operability Affects Versions: 2.0.0 Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0, 0.99.1 Attachments: HBASE-11897-trunk-v1.diff, HBASE-11897-trunk-v2.diff, hbase-11897_v3.patch HBASE-8751 introduces the tables/table-column families config for a replication peer. It's very flexible for practical replication in hbase clusters. But it is easy to make mistakes during add or remove a table/table-column family for a existing peer, especially when the table-cfs is very long, for we need to copy the current table-cfs of the peer first, and then add or remove a table/table-column family to/from the table-cfs, at last set back the table-cfs using the cmd: set_peer_tableCFs. So we implements two new cmds: append_peer_tableCFs and remove_peer_tableCFs to do the operation of adding and removing a table/table-column family. They are useful operation tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11951) Delete stale files from svn site directory and from images directory in git
[ https://issues.apache.org/jira/browse/HBASE-11951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131801#comment-14131801 ] stack commented on HBASE-11951: --- Leave the images unless plain wrong. The images dir has 'other versions' -- bigger versions -- or image editor files from which actual images are derived. We keep them in here so easy to find for those folks who might want to mess with them. Otherwise, excellent cleanup. Thanks misty. +1 on script. Add apache license on head of it like we have on other scripts. Delete stale files from svn site directory and from images directory in git --- Key: HBASE-11951 URL: https://issues.apache.org/jira/browse/HBASE-11951 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Attachments: HBASE-11951.patch, publish_hbase_website.sh -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11959) TestAssignmentManagerOnCluster is flaky
[ https://issues.apache.org/jira/browse/HBASE-11959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131813#comment-14131813 ] Enis Soztutar commented on HBASE-11959: --- +1 if the tests succeed. TestAssignmentManagerOnCluster is flaky --- Key: HBASE-11959 URL: https://issues.apache.org/jira/browse/HBASE-11959 Project: HBase Issue Type: Bug Affects Versions: 0.99.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Attachments: hbase-11959.patch https://builds.apache.org/job/HBase-1.0/178/testReport/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11847) HFile tool should be able to print block headers
[ https://issues.apache.org/jira/browse/HBASE-11847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11847: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to 0.98+. Thanks for the reviews. HFile tool should be able to print block headers Key: HBASE-11847 URL: https://issues.apache.org/jira/browse/HBASE-11847 Project: HBase Issue Type: Improvement Components: HFile Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11847.00.patch, HBASE-11847.01-0.98.patch, HBASE-11847.01.patch Printing the block index is helpful, but sometimes you want to see more info about the blocks themselves. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11847) HFile tool should be able to print block headers
[ https://issues.apache.org/jira/browse/HBASE-11847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11847: - Attachment: HBASE-11847.01-0.98.patch Attaching what was committed to 0.98. HFile tool should be able to print block headers Key: HBASE-11847 URL: https://issues.apache.org/jira/browse/HBASE-11847 Project: HBase Issue Type: Improvement Components: HFile Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11847.00.patch, HBASE-11847.01-0.98.patch, HBASE-11847.01.patch Printing the block index is helpful, but sometimes you want to see more info about the blocks themselves. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11847) HFile tool should be able to print block headers
[ https://issues.apache.org/jira/browse/HBASE-11847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11847: - Release Note: The HFileTool (`hbase hfile`) can print block headers. The flag is -h,--printblockheaders. HFile tool should be able to print block headers Key: HBASE-11847 URL: https://issues.apache.org/jira/browse/HBASE-11847 Project: HBase Issue Type: Improvement Components: HFile Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 2.0.0, 0.98.7, 0.99.1 Attachments: HBASE-11847.00.patch, HBASE-11847.01-0.98.patch, HBASE-11847.01.patch Printing the block index is helpful, but sometimes you want to see more info about the blocks themselves. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11935: --- Attachment: (was: 11935-0.98-v2.txt) ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6 Reporter: Lars Hofhansl Assignee: Jesse Yates Priority: Critical Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11935: --- Attachment: (was: hbase-11935-0.98-v0.patch) ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6 Reporter: Lars Hofhansl Assignee: Jesse Yates Priority: Critical Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11935: --- Attachment: (was: hbase-11935-trunk-v0.patch) ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6 Reporter: Lars Hofhansl Assignee: Jesse Yates Priority: Critical Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11935: --- Attachment: (was: hbase-11935-trunk-v1.patch) ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6 Reporter: Lars Hofhansl Assignee: Jesse Yates Priority: Critical Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11935: --- Attachment: (was: hbase-11935-0.98-v1.patch) ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6 Reporter: Lars Hofhansl Assignee: Jesse Yates Priority: Critical Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11935: --- Attachment: (was: 11935-trunk-v2.txt) ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6 Reporter: Lars Hofhansl Assignee: Jesse Yates Priority: Critical Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11935: --- Status: Open (was: Patch Available) ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.98.6, 0.94.23, 0.99.0, 2.0.0 Reporter: Lars Hofhansl Assignee: Jesse Yates Priority: Critical Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131822#comment-14131822 ] Andrew Purtell commented on HBASE-11935: Removed patches from parent. Will open two subtasks for discussion ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6 Reporter: Lars Hofhansl Assignee: Jesse Yates Priority: Critical Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11869) Support snapshot owner
[ https://issues.apache.org/jira/browse/HBASE-11869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131825#comment-14131825 ] Enis Soztutar commented on HBASE-11869: --- Sorry to come in late, but in regular acl's we got rid of table's owner, because it was an orthogonal model to ACLs, and was not as flexible. Can we maintain the snapshot permissions similar to regular table permissions, and have the same model for tables and snapshots? Support snapshot owner -- Key: HBASE-11869 URL: https://issues.apache.org/jira/browse/HBASE-11869 Project: HBase Issue Type: Improvement Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0 Attachments: HBASE-11869-trunk-v1.diff, HBASE-11869-trunk-v3.diff, HBASE-11869-trunk-v4.diff In current codebase, the table snapshot operations only can be done by the global admin , not by the table admin. There is a multi-tenant hbase cluster, each table has different snapshot policies, eg: do snapshot per week, or snapshot after the new data are imported. We want to release the snapshot permission to each table admin. According to [~mbertozzi]'s suggestion, we implement the snapshot owner feature. * The user with table admin permission can create snapshot and the owner of this snapshot is this user. * The owner of snapshot can delete and restore the snapshot. * Only the user with global admin permission can clone a snapshot, for this operation creates a new table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-11963) Synchronize peer cluster replication connection attempts
Andrew Purtell created HBASE-11963: -- Summary: Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently
[
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131827#comment-14131827
]
bharath v commented on HBASE-11405:
---
Thanks [~j...@cloudera.com] and [~te...@apache.org]. The tests fail because we
abruptly kill the jvm. Ted's version fixes that. I just added a few log lines
as per Jon's suggestion. [~busbey] Mind taking a look at this version?
https://reviews.apache.org/r/25587
Multiple invocations of hbck in parallel disables balancer permanently
---
Key: HBASE-11405
URL: https://issues.apache.org/jira/browse/HBASE-11405
Project: HBase
Issue Type: Bug
Components: Balancer, hbck
Affects Versions: 0.99.0
Reporter: bharath v
Assignee: bharath v
Attachments: 11405-v3.txt, HBASE-11405-trunk-rebased.patch,
HBASE-11405-trunk.patch, HBASE-11405-trunk.patch.1,
hbase-11405.rebase.140911.patch
This is because of the following piece of code in hbck
{code:borderStyle=solid}
boolean oldBalancer = admin.setBalancerRunning(false, true);
try {
onlineConsistencyRepair();
}
finally {
admin.setBalancerRunning(oldBalancer, false);
}
{code}
Newer invocations set oldBalancer to false as it was disabled by previous
invocations and this disables balancer permanently unless its manually turned
on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2
different sessions and you can see that balancer is set to false in the
HMaster logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11935: --- Attachment: (was: hbase-11935-trunk-v2.patch) ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6 Reporter: Lars Hofhansl Assignee: Jesse Yates Priority: Critical Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11962) Port HBASE-11897 Add append and remove peer table-cfs cmds for replication to 0.98
[ https://issues.apache.org/jira/browse/HBASE-11962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131833#comment-14131833 ] stack commented on HBASE-11962: --- Are you doing the backport [~ted_yu]? Is [~liushaohui] incapable of filing his own issues or [~apurtell] the RM, who might want this in 0.98? Port HBASE-11897 Add append and remove peer table-cfs cmds for replication to 0.98 Key: HBASE-11962 URL: https://issues.apache.org/jira/browse/HBASE-11962 Project: HBase Issue Type: Improvement Reporter: Ted Yu Priority: Minor Fix For: 0.98.7 This issue is to backport the commands for appending and removing peer table-cfs for replication to 0.98 Two new commands, append_peer_tableCFs and remove_peer_tableCFs, are added to do the operation of adding and removing a table/table-column family. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-11964) Improve replication source thread handling
Andrew Purtell created HBASE-11964: -- Summary: Improve replication source thread handling Key: HBASE-11964 URL: https://issues.apache.org/jira/browse/HBASE-11964 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Improve replication source thread handling. Limit parallelism when transferring queues. Ensure replication sources terminate properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-11935: -- Fix Version/s: (was: 0.99.1) (was: 0.94.24) (was: 0.98.7) (was: 2.0.0) ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6 Reporter: Lars Hofhansl Assignee: Jesse Yates Priority: Critical We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11963: --- Attachment: HBASE-11963-0.98.patch Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-11963: -- Assignee: Maddineni Sukumar Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Maddineni Sukumar Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch, HBASE-11963.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11963: --- Attachment: HBASE-11963.patch Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Maddineni Sukumar Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch, HBASE-11963.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11963: --- Assignee: Maddineni Sukumar Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Maddineni Sukumar Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch, HBASE-11963.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131838#comment-14131838 ] Lars Hofhansl commented on HBASE-11963: --- +1 Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Maddineni Sukumar Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch, HBASE-11963.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11935: --- Assignee: (was: Jesse Yates) ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6 Reporter: Lars Hofhansl Priority: Critical We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11963: --- Assignee: (was: Maddineni Sukumar) Status: Patch Available (was: Open) Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch, HBASE-11963.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11964) Improve replication source thread handling
[ https://issues.apache.org/jira/browse/HBASE-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11964: --- Attachment: HBASE-11964-0.98.patch Improve replication source thread handling -- Key: HBASE-11964 URL: https://issues.apache.org/jira/browse/HBASE-11964 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11964-0.98.patch Improve replication source thread handling. Limit parallelism when transferring queues. Ensure replication sources terminate properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-11825) Create Connection and ConnectionManager
[
https://issues.apache.org/jira/browse/HBASE-11825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-11825 started by Solomon Duskis.
--
Create Connection and ConnectionManager
---
Key: HBASE-11825
URL: https://issues.apache.org/jira/browse/HBASE-11825
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Solomon Duskis
Priority: Critical
Fix For: 2.0.0, 0.99.1
Attachments: HBASE_11825.patch
This is further cleanup of the HBase interface for 1.0 after implementing the
new Table and Admin interfaces. Following Enis's guidelines in HBASE-10602,
this JIRA will generate a new ConnectionManager to replace HCM and Connection
to replace HConnection.
For more detail, this JIRA intends to implement this portion:
{code}
interface Connection extends Closeable{
Table getTable(), and rest of HConnection methods
getAdmin()
// no deprecated methods (cache related etc)
}
@Deprecated
interface HConnection extends Connection {
@Deprecated
HTableInterface getTable()
// users are encouraged to use Connection
}
class ConnectionManager {
createConnection(Configuration) // not sure whether we want a static
factory method to create connections or a ctor
}
@Deprecated
class HCM extends ConnectionManager {
// users are encouraged to use ConnectionManager
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131854#comment-14131854 ] stack commented on HBASE-11963: --- Seems fine bottlenecking a single RS's query of base attributes of remote cluster. +1 Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Maddineni Sukumar Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch, HBASE-11963.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131859#comment-14131859 ] Lars Hofhansl commented on HBASE-11963: --- The race we saw leaked ZK connection, which eventually caused the parent issue. Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Maddineni Sukumar Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch, HBASE-11963.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11911) Break up tests into more fine grained categories
[
https://issues.apache.org/jira/browse/HBASE-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131860#comment-14131860
]
stack commented on HBASE-11911:
---
I see this in the raw console output. It showed up in previous run IIRC. Let
me run this patch local. I want to see if we run all tests. I don't see why
not. Nor do I see how the failures are related to this change.
{code}
Running org.apache.hadoop.hbase.regionserver.TestFlushRegionEntry
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.694 sec - in
org.apache.hadoop.hbase.regionserver.TestFlushRegionEntry
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/dev-support/test-patch.sh:
line 685: 3689 Killed $MVN clean test -P runAllTests
-D${PROJECT_NAME}PatchProcess
Suspicious java process found - waiting 30s to see if there are just slow to
stop
There are 1 zombie tests, they should have been killed by surefire but survived
BEGIN zombies jstack extract
2014-09-12 05:48:18
Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.51-b03 mixed mode):
Attach Listener daemon prio=10 tid=0x7f1138001000 nid=0x556b waiting on
condition [0x]
java.lang.Thread.State: RUNNABLE
process reaper daemon prio=10 tid=0x7f11906cc800 nid=0x54f2 waiting on
condition [0x7f116e2d6000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for 0x0007e4400578 (a
java.util.concurrent.SynchronousQueue$TransferStack)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{code}
Break up tests into more fine grained categories
Key: HBASE-11911
URL: https://issues.apache.org/jira/browse/HBASE-11911
Project: HBase
Issue Type: Sub-task
Reporter: Alex Newman
Assignee: Alex Newman
Attachments: HBASE-11911-preview.patch, HBASE-11911-v0.patch,
HBASE-11911-v1.patch, HBASE-11911-v2.patch, HBASE-11911-v3.patch,
HBASE-11911-v3.patch, HBASE-11911-v4.patch, HBASE-11911-v4.patch,
HBASE-11911-v5.patch
Wouldn't it be nice to run all of the client tests (client and the ones in
hbase-server) in a single command like.
mvn test -PrunClientTests
how about all of the small tests not having to do with mapReduce
mvn test -PrunSmallTests,!runMapReduceTests
how about breaking up the build into small pieces so it can be run on travis
or circle-ci. If that's what you are into, +1 this patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-11959) TestAssignmentManagerOnCluster is flaky
[ https://issues.apache.org/jira/browse/HBASE-11959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang resolved HBASE-11959. - Resolution: Fixed Fix Version/s: 0.99.1 Hadoop Flags: Reviewed I tested related tests 10 times locally and they are good with this patch. Integrated into branch 1. Thanks. TestAssignmentManagerOnCluster is flaky --- Key: HBASE-11959 URL: https://issues.apache.org/jira/browse/HBASE-11959 Project: HBase Issue Type: Bug Affects Versions: 0.99.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.99.1 Attachments: hbase-11959.patch https://builds.apache.org/job/HBase-1.0/178/testReport/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11897) Add append and remove peer table-cfs cmds for replication
[ https://issues.apache.org/jira/browse/HBASE-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131862#comment-14131862 ] Hudson commented on HBASE-11897: FAILURE: Integrated in HBase-TRUNK #5497 (See [https://builds.apache.org/job/HBase-TRUNK/5497/]) HBASE-11897 Add append and remove peer table-cfs cmds for replication (Liu Shaoqui) (stack: rev c0d4b26872fc97b42ace9ad94d47f402f5428190) * hbase-shell/src/main/ruby/hbase/replication_admin.rb * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWALEntryFilters.java * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeers.java * hbase-shell/src/main/ruby/shell/commands/append_peer_tableCFs.rb * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/replication/TestReplicationAdmin.java * hbase-shell/src/main/ruby/shell/commands/remove_peer_tableCFs.rb * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestPerTableCFReplication.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java * hbase-shell/src/main/ruby/shell.rb * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/TableCfWALEntryFilter.java Add append and remove peer table-cfs cmds for replication - Key: HBASE-11897 URL: https://issues.apache.org/jira/browse/HBASE-11897 Project: HBase Issue Type: Improvement Components: Operability Affects Versions: 2.0.0 Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0, 0.99.1 Attachments: HBASE-11897-trunk-v1.diff, HBASE-11897-trunk-v2.diff, hbase-11897_v3.patch HBASE-8751 introduces the tables/table-column families config for a replication peer. It's very flexible for practical replication in hbase clusters. But it is easy to make mistakes during add or remove a table/table-column family for a existing peer, especially when the table-cfs is very long, for we need to copy the current table-cfs of the peer first, and then add or remove a table/table-column family to/from the table-cfs, at last set back the table-cfs using the cmd: set_peer_tableCFs. So we implements two new cmds: append_peer_tableCFs and remove_peer_tableCFs to do the operation of adding and removing a table/table-column family. They are useful operation tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11964) Improve replication source thread handling
[ https://issues.apache.org/jira/browse/HBASE-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131863#comment-14131863 ] Enis Soztutar commented on HBASE-11964: --- +1. Do you have a master patch? In HBASE-11367, those interfaces changed a bit. Improve replication source thread handling -- Key: HBASE-11964 URL: https://issues.apache.org/jira/browse/HBASE-11964 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11964-0.98.patch Improve replication source thread handling. Limit parallelism when transferring queues. Ensure replication sources terminate properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11964) Improve replication source thread handling
[ https://issues.apache.org/jira/browse/HBASE-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131868#comment-14131868 ] Andrew Purtell commented on HBASE-11964: I am working on the master port right now Improve replication source thread handling -- Key: HBASE-11964 URL: https://issues.apache.org/jira/browse/HBASE-11964 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11964-0.98.patch Improve replication source thread handling. Limit parallelism when transferring queues. Ensure replication sources terminate properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11964) Improve replication source thread handling
[ https://issues.apache.org/jira/browse/HBASE-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11964: --- Attachment: HBASE-11964.patch Patch for trunk Improve replication source thread handling -- Key: HBASE-11964 URL: https://issues.apache.org/jira/browse/HBASE-11964 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11964-0.98.patch, HBASE-11964.patch Improve replication source thread handling. Limit parallelism when transferring queues. Ensure replication sources terminate properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11893) RowTooBigException should be in hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-11893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-11893: Attachment: HBASE-11893.v3.patch Oops, somehow the patch got messed up when I was redoing the exporting. Attached fixed one. Thanks for noticing. RowTooBigException should be in hbase-client module --- Key: HBASE-11893 URL: https://issues.apache.org/jira/browse/HBASE-11893 Project: HBase Issue Type: Bug Components: Client Reporter: Sean Busbey Assignee: Mikhail Antonov Priority: Minor Labels: beginner Attachments: HBASE-11893.patch, HBASE-11893.v2.patch, HBASE-11893.v3.patch RowTooBigException is Public get thrown from client calls. It should be in hbase-client instead of hbase-server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11893) RowTooBigException should be in hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-11893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-11893: Status: Patch Available (was: Open) RowTooBigException should be in hbase-client module --- Key: HBASE-11893 URL: https://issues.apache.org/jira/browse/HBASE-11893 Project: HBase Issue Type: Bug Components: Client Reporter: Sean Busbey Assignee: Mikhail Antonov Priority: Minor Labels: beginner Attachments: HBASE-11893.patch, HBASE-11893.v2.patch, HBASE-11893.v3.patch RowTooBigException is Public get thrown from client calls. It should be in hbase-client instead of hbase-server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11893) RowTooBigException should be in hbase-client module
[ https://issues.apache.org/jira/browse/HBASE-11893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-11893: Status: Open (was: Patch Available) RowTooBigException should be in hbase-client module --- Key: HBASE-11893 URL: https://issues.apache.org/jira/browse/HBASE-11893 Project: HBase Issue Type: Bug Components: Client Reporter: Sean Busbey Assignee: Mikhail Antonov Priority: Minor Labels: beginner Attachments: HBASE-11893.patch, HBASE-11893.v2.patch, HBASE-11893.v3.patch RowTooBigException is Public get thrown from client calls. It should be in hbase-client instead of hbase-server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131887#comment-14131887 ] Andrew Purtell commented on HBASE-11963: +1 Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Maddineni Sukumar Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch, HBASE-11963.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131888#comment-14131888 ] Andrew Purtell edited comment on HBASE-11963 at 9/12/14 6:34 PM: - Ok, will commit this later today unless objection was (Author: apurtell): Ok, will commit this shortly unless objection Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Maddineni Sukumar Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch, HBASE-11963.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131888#comment-14131888 ] Andrew Purtell commented on HBASE-11963: Ok, will commit this shortly unless objection Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Maddineni Sukumar Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch, HBASE-11963.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11935) ZooKeeper connection storm after queue failover with slave cluster down
[ https://issues.apache.org/jira/browse/HBASE-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131889#comment-14131889 ] Andrew Purtell commented on HBASE-11935: Can be resolved after subtasks ZooKeeper connection storm after queue failover with slave cluster down --- Key: HBASE-11935 URL: https://issues.apache.org/jira/browse/HBASE-11935 Project: HBase Issue Type: Bug Affects Versions: 0.99.0, 2.0.0, 0.94.23, 0.98.6 Reporter: Lars Hofhansl Priority: Critical We just ran into a production incident with TCP SYN storms on port 2181 (zookeeper). In our case the slave cluster was not running. When we bounced the primary cluster we saw an unbounded number of failover threads all hammering the hosts on the slave ZK machines (which did not run ZK at the time)... Causing overall degradation of network performance between datacenters. Looking at the code we noticed that the thread pool handling of the Failover workers was probably unintended. Patch coming soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11949) Setting hfile.block.cache.size=0 doesn't actually disable blockcache
[ https://issues.apache.org/jira/browse/HBASE-11949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131893#comment-14131893 ] Nick Dimiduk commented on HBASE-11949: -- TestAdmin and TestScannerTimeout both pas for me locally. Setting hfile.block.cache.size=0 doesn't actually disable blockcache Key: HBASE-11949 URL: https://issues.apache.org/jira/browse/HBASE-11949 Project: HBase Issue Type: Bug Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 2.0.0, 0.99.1 Attachments: HBASE-11949.00.patch [~stack] noticed this one over on HBASE-11845. The provided patched worked as intended on 0.98, but not on branch-1 or master. Marking as minor because we highly encourage users not to do this anyway (it's just a convenience for tools). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11825) Create Connection and ConnectionManager
[
https://issues.apache.org/jira/browse/HBASE-11825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131897#comment-14131897
]
Enis Soztutar commented on HBASE-11825:
---
Thanks Solomon, Carter for working on this.
Can you please change below to be Connection#getTable(). I think in my original
patch, I started with Connection#createTable() and Connection#createAdmin()
instead of getTable(), but changed it because createTable() is creating a
Table.java object rather than creating an actual table for data, ie
Admin.createTable().
{code}
+ * From this {@link Connection} {@link Table} implementations are retrieved
+ * with {@link Connection#createTable(byte[])}. Example:
{code}
bq. Admin.getConnection() returns a Connection and HBaseAdmin.getConnection()
returns an HConnection.
This is good. Will keep BC. Admin is a new interface.
maybe remove this line:
{code}
+ * listTables() kind of methods should be delegated to admin? TODO
{code}
bq. I had to revert some of the Admin - HBaseAdmin changes after I changed
Admin's getConnection method to return a Connection instead of an HConnection
Ok, this should be fine for now.
This is pretty important. I would love the other devs to take a look as well.
Create Connection and ConnectionManager
---
Key: HBASE-11825
URL: https://issues.apache.org/jira/browse/HBASE-11825
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Solomon Duskis
Priority: Critical
Fix For: 2.0.0, 0.99.1
Attachments: HBASE_11825.patch
This is further cleanup of the HBase interface for 1.0 after implementing the
new Table and Admin interfaces. Following Enis's guidelines in HBASE-10602,
this JIRA will generate a new ConnectionManager to replace HCM and Connection
to replace HConnection.
For more detail, this JIRA intends to implement this portion:
{code}
interface Connection extends Closeable{
Table getTable(), and rest of HConnection methods
getAdmin()
// no deprecated methods (cache related etc)
}
@Deprecated
interface HConnection extends Connection {
@Deprecated
HTableInterface getTable()
// users are encouraged to use Connection
}
class ConnectionManager {
createConnection(Configuration) // not sure whether we want a static
factory method to create connections or a ctor
}
@Deprecated
class HCM extends ConnectionManager {
// users are encouraged to use ConnectionManager
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-11949) Setting hfile.block.cache.size=0 doesn't actually disable blockcache
[ https://issues.apache.org/jira/browse/HBASE-11949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11949: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-1 and master. Thanks folks. Setting hfile.block.cache.size=0 doesn't actually disable blockcache Key: HBASE-11949 URL: https://issues.apache.org/jira/browse/HBASE-11949 Project: HBase Issue Type: Bug Components: regionserver Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 2.0.0, 0.99.1 Attachments: HBASE-11949.00.patch [~stack] noticed this one over on HBASE-11845. The provided patched worked as intended on 0.98, but not on branch-1 or master. Marking as minor because we highly encourage users not to do this anyway (it's just a convenience for tools). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11963) Synchronize peer cluster replication connection attempts
[ https://issues.apache.org/jira/browse/HBASE-11963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131900#comment-14131900 ] Lars Hofhansl commented on HBASE-11963: --- I have a 0.94 patch too, won't attach now in order to not throw off HadoopQA. Synchronize peer cluster replication connection attempts Key: HBASE-11963 URL: https://issues.apache.org/jira/browse/HBASE-11963 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Maddineni Sukumar Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11963-0.98.patch, HBASE-11963.patch Synchronize peer cluster connection attempts to avoid races and rate limit connections when multiple replication sources try to connect to the peer cluster. If the peer cluster is down we can get out of control over time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11825) Create Connection and ConnectionManager
[
https://issues.apache.org/jira/browse/HBASE-11825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131903#comment-14131903
]
stack commented on HBASE-11825:
---
I like this cleanup in the new Admin Interface:
- HConnection getConnection();
+ Connection getConnection();
Nit: read over the Connection javadoc -- which is really nice and clean BTW --
because there a few fixes ('names H is' should be 'names H are'... 'an Table'
when should be 'a Table', etc)
Nit: should this be in the Interface?
+ boolean isClosed();
Make close idempotent instead?
Is this right?
+ * From this {@link Connection} {@link Table} implementations are retrieved
+ * with {@link Connection#createTable(byte[])}.
There is no createTable in your nice new Connection Interface?
Refer to your class comment rather than reproduce the below in your javadoc
especially as class comment encourages try/finally and this example is absent
it:
{code}
+ * Connection connection = ConnectionFactory.createConnection(conf);
+ * Table table = connection.getTable(mytable);
+ * table.get(...);
+ * ...
+ * table.close();
+ * connection.close();
{code}
This is good:
+@Deprecated
+public class HConnectionManager extends ConnectionFactory {
I thought HCM was internal-only anyways but yeah, it ugly and I like this
effort to better hide the mess.
Regards the restore of HBaseAdmin, it looks to be in test code only. That is
fine IMO.
[~lhofhansl] I know you are busy but you might take a quick look here given
they seem to address a longtime peeve of yours.
Create Connection and ConnectionManager
---
Key: HBASE-11825
URL: https://issues.apache.org/jira/browse/HBASE-11825
Project: HBase
Issue Type: Improvement
Reporter: Carter
Assignee: Solomon Duskis
Priority: Critical
Fix For: 2.0.0, 0.99.1
Attachments: HBASE_11825.patch
This is further cleanup of the HBase interface for 1.0 after implementing the
new Table and Admin interfaces. Following Enis's guidelines in HBASE-10602,
this JIRA will generate a new ConnectionManager to replace HCM and Connection
to replace HConnection.
For more detail, this JIRA intends to implement this portion:
{code}
interface Connection extends Closeable{
Table getTable(), and rest of HConnection methods
getAdmin()
// no deprecated methods (cache related etc)
}
@Deprecated
interface HConnection extends Connection {
@Deprecated
HTableInterface getTable()
// users are encouraged to use Connection
}
class ConnectionManager {
createConnection(Configuration) // not sure whether we want a static
factory method to create connections or a ctor
}
@Deprecated
class HCM extends ConnectionManager {
// users are encouraged to use ConnectionManager
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11964) Improve replication source thread handling
[ https://issues.apache.org/jira/browse/HBASE-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131904#comment-14131904 ] Enis Soztutar commented on HBASE-11964: --- +1. Improve replication source thread handling -- Key: HBASE-11964 URL: https://issues.apache.org/jira/browse/HBASE-11964 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Fix For: 2.0.0, 0.98.7, 0.94.24, 0.99.1 Attachments: HBASE-11964-0.98.patch, HBASE-11964.patch Improve replication source thread handling. Limit parallelism when transferring queues. Ensure replication sources terminate properly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-7767) Get rid of ZKTable, and table enable/disable state in ZK
[
https://issues.apache.org/jira/browse/HBASE-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131917#comment-14131917
]
Hadoop QA commented on HBASE-7767:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12668366/HBASE-7767.patch
against trunk revision .
ATTACHMENT ID: 12668366
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 45 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the trunk's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.access.TestAccessController2
org.apache.hadoop.hbase.master.TestMasterFailover
org.apache.hadoop.hbase.security.visibility.TestEnforcingScanLabelGenerator
org.apache.hadoop.hbase.client.TestReplicaWithCluster
org.apache.hadoop.hbase.replication.TestPerTableCFReplication
org.apache.hadoop.hbase.security.access.TestTablePermissions
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10860//console
This message is automatically generated.
Get rid of ZKTable, and table enable/disable state in ZK
-
Key: HBASE-7767
URL: https://issues.apache.org/jira/browse/HBASE-7767
Project: HBase
Issue Type: Sub-task
Components: Zookeeper
Affects Versions: 2.0.0
Reporter: Enis Soztutar
Assignee: Andrey Stepachev
Attachments: HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch,
HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch,
HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch,
HBASE-7767.patch
As discussed table state in zookeeper for enable/disable state breaks our
zookeeper contract. It is also very intrusive, used from the client side,
master and region servers. We should get rid of it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-7767) Get rid of ZKTable, and table enable/disable state in ZK
[ https://issues.apache.org/jira/browse/HBASE-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131930#comment-14131930 ] stack commented on HBASE-7767: -- Are they your failures [~octo47]? TestPerTableCFReplication may be because of a commit just made. I'll check that one. Get rid of ZKTable, and table enable/disable state in ZK - Key: HBASE-7767 URL: https://issues.apache.org/jira/browse/HBASE-7767 Project: HBase Issue Type: Sub-task Components: Zookeeper Affects Versions: 2.0.0 Reporter: Enis Soztutar Assignee: Andrey Stepachev Attachments: HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch, HBASE-7767.patch As discussed table state in zookeeper for enable/disable state breaks our zookeeper contract. It is also very intrusive, used from the client side, master and region servers. We should get rid of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11897) Add append and remove peer table-cfs cmds for replication
[ https://issues.apache.org/jira/browse/HBASE-11897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131934#comment-14131934 ] Hudson commented on HBASE-11897: FAILURE: Integrated in HBase-1.0 #179 (See [https://builds.apache.org/job/HBase-1.0/179/]) HBASE-11897 Add append and remove peer table-cfs cmds for replication (Liu Shaoqui) (stack: rev 511b20a227fca5ba3d283caf4f847c852d049451) * hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeerZKImpl.java * hbase-shell/src/main/ruby/hbase/replication_admin.rb * hbase-shell/src/main/ruby/shell.rb * hbase-shell/src/main/ruby/shell/commands/append_peer_tableCFs.rb * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeers.java * hbase-shell/src/main/ruby/shell/commands/remove_peer_tableCFs.rb * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeersZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/replication/TableCfWALEntryFilter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationWALEntryFilters.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestPerTableCFReplication.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/replication/TestReplicationAdmin.java * hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java Add append and remove peer table-cfs cmds for replication - Key: HBASE-11897 URL: https://issues.apache.org/jira/browse/HBASE-11897 Project: HBase Issue Type: Improvement Components: Operability Affects Versions: 2.0.0 Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0, 0.99.1 Attachments: HBASE-11897-trunk-v1.diff, HBASE-11897-trunk-v2.diff, hbase-11897_v3.patch HBASE-8751 introduces the tables/table-column families config for a replication peer. It's very flexible for practical replication in hbase clusters. But it is easy to make mistakes during add or remove a table/table-column family for a existing peer, especially when the table-cfs is very long, for we need to copy the current table-cfs of the peer first, and then add or remove a table/table-column family to/from the table-cfs, at last set back the table-cfs using the cmd: set_peer_tableCFs. So we implements two new cmds: append_peer_tableCFs and remove_peer_tableCFs to do the operation of adding and removing a table/table-column family. They are useful operation tools. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11911) Break up tests into more fine grained categories
[ https://issues.apache.org/jira/browse/HBASE-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131950#comment-14131950 ] Alex Newman commented on HBASE-11911: - [~stack] Sounds good. I don't mind rebasing daily. Thanks for doing the legwork. Break up tests into more fine grained categories Key: HBASE-11911 URL: https://issues.apache.org/jira/browse/HBASE-11911 Project: HBase Issue Type: Sub-task Reporter: Alex Newman Assignee: Alex Newman Attachments: HBASE-11911-preview.patch, HBASE-11911-v0.patch, HBASE-11911-v1.patch, HBASE-11911-v2.patch, HBASE-11911-v3.patch, HBASE-11911-v3.patch, HBASE-11911-v4.patch, HBASE-11911-v4.patch, HBASE-11911-v5.patch Wouldn't it be nice to run all of the client tests (client and the ones in hbase-server) in a single command like. mvn test -PrunClientTests how about all of the small tests not having to do with mapReduce mvn test -PrunSmallTests,!runMapReduceTests how about breaking up the build into small pieces so it can be run on travis or circle-ci. If that's what you are into, +1 this patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11911) Break up tests into more fine grained categories
[ https://issues.apache.org/jira/browse/HBASE-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131955#comment-14131955 ] stack commented on HBASE-11911: --- [~posix4e] Hang on. Its broke again but I fixed it local. Let me see if I can get a run that looks like the others... will commit then. Break up tests into more fine grained categories Key: HBASE-11911 URL: https://issues.apache.org/jira/browse/HBASE-11911 Project: HBase Issue Type: Sub-task Reporter: Alex Newman Assignee: Alex Newman Attachments: HBASE-11911-preview.patch, HBASE-11911-v0.patch, HBASE-11911-v1.patch, HBASE-11911-v2.patch, HBASE-11911-v3.patch, HBASE-11911-v3.patch, HBASE-11911-v4.patch, HBASE-11911-v4.patch, HBASE-11911-v5.patch Wouldn't it be nice to run all of the client tests (client and the ones in hbase-server) in a single command like. mvn test -PrunClientTests how about all of the small tests not having to do with mapReduce mvn test -PrunSmallTests,!runMapReduceTests how about breaking up the build into small pieces so it can be run on travis or circle-ci. If that's what you are into, +1 this patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-11965) Optimize locking in HRegion
Vladimir Rodionov created HBASE-11965: - Summary: Optimize locking in HRegion Key: HBASE-11965 URL: https://issues.apache.org/jira/browse/HBASE-11965 Project: HBase Issue Type: Bug Components: Performance Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov Described in this thread: http://mail-archives.apache.org/mod_mbox/hbase-dev/201409.mbox/%3CCAAg3a2qmWJtQbdAk7PrX%2BW8SZWxsYhNggM5f2RNkGTXUri34YQ%40mail.gmail.com%3E *startRegionOperation* and *closeRegionOperation* in HRegion acquires and releases region-wide lock. The locking happens for every mutation, read and scan next. This is a serious bottleneck if we do: * Mutli - get on a same region. * Run multiple scanners on same region. * Do scan during compaction. * Access region simultaneously from different threads (most generic case) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
