[jira] [Updated] (HBASE-11671) TestEndToEndSplitTransaction fails on master
[
https://issues.apache.org/jira/browse/HBASE-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-11671:
Status: Patch Available (was: Open)
TestEndToEndSplitTransaction fails on master
Key: HBASE-11671
URL: https://issues.apache.org/jira/browse/HBASE-11671
Project: HBase
Issue Type: Bug
Components: Region Assignment, test
Affects Versions: 0.99.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 0.99.0
Attachments: HBASE-11671.patch
#testMasterOpsWhileSplitting() fails, seems after enabling ZK-less assignment.
Here're the only log snippets which seem relevant.
{quote}
java.io.IOException: Failed to report opened region to master:
TestSplit,lll,1407218018712.030c51654a5cbe9e0489a50762bb9075.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1731)
at
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.testMasterOpsWhileSplitting(TestEndToEndSplitTransaction.java:130)
{quote}
{quote}
java.io.IOException: Cannot append; log is closed
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1146)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.appendNoSync(FSHLog.java:1120)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:266)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1211)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1158)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1514)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:478)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11671) TestEndToEndSplitTransaction fails on master
[
https://issues.apache.org/jira/browse/HBASE-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-11671:
Attachment: HBASE-11671.patch
attached one-liner patch which fixes the test by setting usezk=true in
config, just to get the test passing, looking further in the test.
TestEndToEndSplitTransaction fails on master
Key: HBASE-11671
URL: https://issues.apache.org/jira/browse/HBASE-11671
Project: HBase
Issue Type: Bug
Components: Region Assignment, test
Affects Versions: 0.99.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 0.99.0
Attachments: HBASE-11671.patch
#testMasterOpsWhileSplitting() fails, seems after enabling ZK-less assignment.
Here're the only log snippets which seem relevant.
{quote}
java.io.IOException: Failed to report opened region to master:
TestSplit,lll,1407218018712.030c51654a5cbe9e0489a50762bb9075.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1731)
at
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.testMasterOpsWhileSplitting(TestEndToEndSplitTransaction.java:130)
{quote}
{quote}
java.io.IOException: Cannot append; log is closed
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1146)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.appendNoSync(FSHLog.java:1120)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:266)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1211)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1158)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1514)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:478)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11617) incorrect AgeOfLastAppliedOp and AgeOfLastShippedOp in replication Metrics when no new replication OP
[
https://issues.apache.org/jira/browse/HBASE-11617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085856#comment-14085856
]
Lars Hofhansl commented on HBASE-11617:
---
Sorry... Was occupied with another issue.
incorrect AgeOfLastAppliedOp and AgeOfLastShippedOp in replication Metrics
when no new replication OP
--
Key: HBASE-11617
URL: https://issues.apache.org/jira/browse/HBASE-11617
Project: HBase
Issue Type: Bug
Components: Replication
Affects Versions: 0.98.2
Reporter: Demai Ni
Assignee: Demai Ni
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: HBASE-11617-master-v1.patch
AgeOfLastAppliedOp in MetricsSink.java is to indicate the time an edit sat in
the 'replication queue' before it got replicated(aka applied)
{code}
/**
* Set the age of the last applied operation
*
* @param timestamp The timestamp of the last operation applied.
* @return the age that was set
*/
public long setAgeOfLastAppliedOp(long timestamp) {
lastTimestampForAge = timestamp;
long age = System.currentTimeMillis() - lastTimestampForAge;
rms.setGauge(SINK_AGE_OF_LAST_APPLIED_OP, age);
return age;
}
{code}
In the following scenario:
1) at 7:00am a sink op is applied, and the SINK_AGE_OF_LAST_APPLIED_OP is
set for example 100ms;
2) and then NO new Sink op occur.
3) when a refreshAgeOfLastAppliedOp() is invoked at 8:00am. Instead of
return the 100ms, the AgeOfLastAppliedOp become 1hour + 100ms,
It was because that refreshAgeOfLastAppliedOp() get invoked periodically by
getStats().
proposed fix:
{code}
---
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSink.java
+++
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSink.java
@@ -35,6 +35,7 @@ public class MetricsSink {
private MetricsReplicationSource rms;
private long lastTimestampForAge = System.currentTimeMillis();
+ private long age = 0;
public MetricsSink() {
rms =
CompatibilitySingletonFactory.getInstance(MetricsReplicationSource.class);
@@ -47,8 +48,12 @@ public class MetricsSink {
* @return the age that was set
*/
public long setAgeOfLastAppliedOp(long timestamp) {
-lastTimestampForAge = timestamp;
-long age = System.currentTimeMillis() - lastTimestampForAge;
+if (lastTimestampForAge != timestamp) {
+ lastTimestampForAge = timestamp;
+ this.age = System.currentTimeMillis() - lastTimestampForAge;
+} else {
+ this.age = 0;
+}
rms.setGauge(SINK_AGE_OF_LAST_APPLIED_OP, age);
return age;
}
{code}
detail discussion in [dev@hbase |
http://mail-archives.apache.org/mod_mbox/hbase-dev/201407.mbox/%3CCAOEq2C5BKMXAM2Fv4LGVb_Ktek-Pm%3DhjOi33gSHX-2qHqAou6w%40mail.gmail.com%3E
]
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11642) EOL 0.96
[ https://issues.apache.org/jira/browse/HBASE-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085866#comment-14085866 ] Lars Hofhansl commented on HBASE-11642: --- [~ishanc] In the end this an open source project. As long as there is a maintainer there'll be updates. :) The reasoning for retiring 0.96 is that everyone should rather go to 0.98 (you can do a rolling upgrade from 0.96 to 0.98, no data upgrade required). If we had semantic versioning going from 0.96 to 0.98 would something like going from version 2.0.x to 2.1.x (i.e. a minor version update). We're just saying: No more patches for the 2.0 code line, please upgrade to 2.1.x for bugfixes. 0.94 is the prior major version. Retiring that in favor of 0.98 would be like going from 1.0.x to 2.x.y, with no rolling upgrades, no client-server protocol compatibility, and a data upgrade step and new version of Hadoop (likely) required. So we'd be maintaining the equivalent of 1.x.y, and 2.x.y. Eventually 0.94 will be retired (i.e. nobody will contribute patches to it any more). If we want save the work maintaining 0.94 or nobody is using it anymore anyway we can retire it soon. If we want to keep maintaining it for folks who cannot risk upgrading their data we can do so too. Why are you asking? You feel we should retire 0.94 as well? Or you're worried we might retire it? EOL 0.96 Key: HBASE-11642 URL: https://issues.apache.org/jira/browse/HBASE-11642 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Do the work to EOL 0.96. + No more patches on 0.96 + Remove 0.96 from downloads. + If user has issue with 0.96 and needs fix, fix it in 0.98 and have the user upgrade to get the fix. + Write email to user list stating 0.96 has been EOL'd September 1st? And add notice to refguide. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11671) TestEndToEndSplitTransaction fails on master
[
https://issues.apache.org/jira/browse/HBASE-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085873#comment-14085873
]
Mikhail Antonov commented on HBASE-11671:
-
[~jxiang] As I'm looking at it, when HRS.postOpenDeployTasks is called, if
zk-less assignment is on, it results in RPC call to master to move region to
OPENED state, but in AM#onRegionTransition().. state machine seems to prohibit
transitions from SPLITTING_NEW (which is the state master has) to OPENED. What
do you think?
TestEndToEndSplitTransaction fails on master
Key: HBASE-11671
URL: https://issues.apache.org/jira/browse/HBASE-11671
Project: HBase
Issue Type: Bug
Components: Region Assignment, test
Affects Versions: 0.99.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 0.99.0
Attachments: HBASE-11671.patch
#testMasterOpsWhileSplitting() fails, seems after enabling ZK-less assignment.
Here're the only log snippets which seem relevant.
{quote}
java.io.IOException: Failed to report opened region to master:
TestSplit,lll,1407218018712.030c51654a5cbe9e0489a50762bb9075.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1731)
at
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.testMasterOpsWhileSplitting(TestEndToEndSplitTransaction.java:130)
{quote}
{quote}
java.io.IOException: Cannot append; log is closed
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1146)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.appendNoSync(FSHLog.java:1120)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:266)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1211)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1158)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1514)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:478)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11671) TestEndToEndSplitTransaction fails on master
[
https://issues.apache.org/jira/browse/HBASE-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085878#comment-14085878
]
Hadoop QA commented on HBASE-11671:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12659827/HBASE-11671.patch
against trunk revision .
ATTACHMENT ID: 12659827
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.ipc.TestIPC
org.apache.hadoop.hbase.master.TestClockSkewDetection
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10292//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10292//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10292//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10292//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10292//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10292//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10292//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10292//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10292//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10292//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10292//console
This message is automatically generated.
TestEndToEndSplitTransaction fails on master
Key: HBASE-11671
URL: https://issues.apache.org/jira/browse/HBASE-11671
Project: HBase
Issue Type: Bug
Components: Region Assignment, test
Affects Versions: 0.99.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 0.99.0
Attachments: HBASE-11671.patch
#testMasterOpsWhileSplitting() fails, seems after enabling ZK-less assignment.
Here're the only log snippets which seem relevant.
{quote}
java.io.IOException: Failed to report opened region to master:
TestSplit,lll,1407218018712.030c51654a5cbe9e0489a50762bb9075.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1731)
at
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.testMasterOpsWhileSplitting(TestEndToEndSplitTransaction.java:130)
{quote}
{quote}
java.io.IOException: Cannot append; log is closed
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1146)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.appendNoSync(FSHLog.java:1120)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:266)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1211)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1158)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1514)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:478)
at
[jira] [Updated] (HBASE-11667) Simplify ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-11667:
---
Status: Open (was: Patch Available)
Simplify ClientScanner logic for NSREs.
---
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11667) Simplify ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-11667:
--
Attachment: 11667-doc-0.94.txt
How this for comment? (0.94)
Simplify ClientScanner logic for NSREs.
---
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11667) Simplify ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085881#comment-14085881
]
Lars Hofhansl commented on HBASE-11667:
---
I also wonder now how this would behave with a filter that filters most (or
all) KVs for a region in the case when we have a stale cache due to splits...
Since the ClientScanner has nothing to go by it would redo the scan of the
entire prior region.
Simplify ClientScanner logic for NSREs.
---
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-11667:
--
Attachment: (was: 11667-doc-0.94.txt)
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-trunk.txt, HBASE-11667-0.98.patch,
IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-11667:
--
Summary: Comment ClientScanner logic for NSREs. (was: Simplify
ClientScanner logic for NSREs.)
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-trunk.txt, HBASE-11667-0.98.patch,
IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11667) Simplify ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-11667:
--
Priority: Minor (was: Major)
Simplify ClientScanner logic for NSREs.
---
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-trunk.txt, HBASE-11667-0.98.patch,
IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085886#comment-14085886
]
Lars Hofhansl commented on HBASE-11667:
---
And I think we should comment [~apurtell]'s integration test.
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-trunk.txt, HBASE-11667-0.98.patch,
IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11642) EOL 0.96
[ https://issues.apache.org/jira/browse/HBASE-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085885#comment-14085885 ] Ishan Chhabra commented on HBASE-11642: --- [~lhofhansl], I agree with your reasoning of maintaining 0.94 a little longer (given the incompatibility and a stop the world upgrade process), but I am worried we might end up in a Python 2.x 3.x like scenario. Declaring an EOL for 0.94 might be a good way to trigger upgrades for many clients who have not done it yet and will bring more focus to the community (I see many JIRAs with won't backport for 0.94, but some patches attached, and people will keep on coming back to the mailing lists seeking help for these and other issues). EOL 0.96 Key: HBASE-11642 URL: https://issues.apache.org/jira/browse/HBASE-11642 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Do the work to EOL 0.96. + No more patches on 0.96 + Remove 0.96 from downloads. + If user has issue with 0.96 and needs fix, fix it in 0.98 and have the user upgrade to get the fix. + Write email to user list stating 0.96 has been EOL'd September 1st? And add notice to refguide. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-11667:
--
Attachment: 11667-doc-0.94.txt
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11672) Add IntegrationTestBigLinkedListWithRegionMovement
Andrew Purtell created HBASE-11672: -- Summary: Add IntegrationTestBigLinkedListWithRegionMovement Key: HBASE-11672 URL: https://issues.apache.org/jira/browse/HBASE-11672 Project: HBase Issue Type: Test Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE-11672.patch An integration test that extends ITBLL with a fixed monkey policy that moves a random region of the table very often. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11672) Add IntegrationTestBigLinkedListWithRegionMovement
[ https://issues.apache.org/jira/browse/HBASE-11672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11672: --- Status: Patch Available (was: Open) Add IntegrationTestBigLinkedListWithRegionMovement -- Key: HBASE-11672 URL: https://issues.apache.org/jira/browse/HBASE-11672 Project: HBase Issue Type: Test Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE-11672.patch An integration test that extends ITBLL with a fixed monkey policy that moves a random region of the table very often. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085891#comment-14085891
]
Andrew Purtell commented on HBASE-11667:
bq. I also wonder now how this would behave with a filter that filters most (or
all) KVs for a region in the case when we have a stale cache due to splits...
Since the ClientScanner has nothing to go by it would redo the scan of the
entire prior region.
We should come up with a test that finds out. Might be a bug issue there.
bq. And I think we should comment Andrew Purtell's integration test.
See HBASE-11672
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11672) Add IntegrationTestBigLinkedListWithRegionMovement
[ https://issues.apache.org/jira/browse/HBASE-11672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11672: --- Attachment: HBASE-11672.patch Add IntegrationTestBigLinkedListWithRegionMovement -- Key: HBASE-11672 URL: https://issues.apache.org/jira/browse/HBASE-11672 Project: HBase Issue Type: Test Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE-11672.patch An integration test that extends ITBLL with a fixed monkey policy that moves a random region of the table very often. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11642) EOL 0.96
[ https://issues.apache.org/jira/browse/HBASE-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085902#comment-14085902 ] Lars Hofhansl commented on HBASE-11642: --- Is Python 2.x vs 3.x, Linux Kernel 2.x vs 3.x, Java 7 vs 8, etc, necessarily a bad thing? The older main versions exist because folks cannot (or do not want to) upgrade. I hear the let's force an upgrade by unsupporting version X from non-practicing entities a lot - i.e. those that actually do not use the product themselves. (not saying that's the case with you :) ) As chance has it my organization is upgrading to 0.98 as we speak so it's not an issue for me, but I imagine if we didn't, we certainly wouldn't want to be a situation where we're on our own with a heavy investment in a certain infrastructure (in terms of H/W, operations, customer commitments, etc), and only a destructive one-way upgrade path that includes downtime. (We're doing this now before the *big* rollout in order to avoid these head-aches for a bit.) EOL 0.96 Key: HBASE-11642 URL: https://issues.apache.org/jira/browse/HBASE-11642 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Do the work to EOL 0.96. + No more patches on 0.96 + Remove 0.96 from downloads. + If user has issue with 0.96 and needs fix, fix it in 0.98 and have the user upgrade to get the fix. + Write email to user list stating 0.96 has been EOL'd September 1st? And add notice to refguide. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085904#comment-14085904
]
Lars Hofhansl commented on HBASE-11667:
---
I meant *commit* Andrew's test...
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085913#comment-14085913
]
Lars Hofhansl commented on HBASE-11667:
---
bq. We should come up with a test that finds out. Might be a bug issue there.
Not a bug, I think, just unnecessary work. I also do not see any other way of
doing this. When a region moves even from a logical viewpoint you *need* to
know where to start off again or you have to start from last known position
(which is the start key of the last region).
I suppose every scan could always send the last row seen as extra metadata at
the end of the RPC. The ClientScanner would record that, but not pass it on to
the caller. That would still break the Phoenix scenario that started this
discussion though, as the region observer does not send a useful row key back
at all.
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085919#comment-14085919
]
Andrew Purtell commented on HBASE-11667:
bq. I suppose every scan could always send the last row seen as extra metadata
at the end of the RPC.
That wouldn't help. It's suspicious that the client relies on the server
telling it state it could track locally (or am I missing something?). In the
Phoenix case the server lies because a coprocessor changes something, and the
client cannot recover, but could/should?
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085931#comment-14085931
]
Lars Hofhansl commented on HBASE-11667:
---
bq. It's suspicious that the client relies on the server telling it state it
could track locally (or am I missing something?).
The client cannot track what the server is doing unless the server tells it
what it did (i.e. how far it got with its scan). I don't think we can recover
if there is no way to know which state to recover to.
All the client can know without help from the server (this particular scan) if
last startkey of the last region. I tried to only use that information, but it
turns out that does not work. Interesting problem :)
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11672) Add IntegrationTestBigLinkedListWithRegionMovement
[
https://issues.apache.org/jira/browse/HBASE-11672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085933#comment-14085933
]
Hadoop QA commented on HBASE-11672:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12659839/HBASE-11672.patch
against trunk revision .
ATTACHMENT ID: 12659839
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestClockSkewDetection
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.ipc.TestIPC
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10293//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10293//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10293//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10293//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10293//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10293//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10293//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10293//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10293//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10293//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10293//console
This message is automatically generated.
Add IntegrationTestBigLinkedListWithRegionMovement
--
Key: HBASE-11672
URL: https://issues.apache.org/jira/browse/HBASE-11672
Project: HBase
Issue Type: Test
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.98.6
Attachments: HBASE-11672.patch
An integration test that extends ITBLL with a fixed monkey policy that moves
a random region of the table very often.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11671) TestEndToEndSplitTransaction fails on master
[
https://issues.apache.org/jira/browse/HBASE-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-11671:
Status: Open (was: Patch Available)
TestEndToEndSplitTransaction fails on master
Key: HBASE-11671
URL: https://issues.apache.org/jira/browse/HBASE-11671
Project: HBase
Issue Type: Bug
Components: Region Assignment, test
Affects Versions: 0.99.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 0.99.0
Attachments: HBASE-11671-v2.patch, HBASE-11671.patch
#testMasterOpsWhileSplitting() fails, seems after enabling ZK-less assignment.
Here're the only log snippets which seem relevant.
{quote}
java.io.IOException: Failed to report opened region to master:
TestSplit,lll,1407218018712.030c51654a5cbe9e0489a50762bb9075.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1731)
at
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.testMasterOpsWhileSplitting(TestEndToEndSplitTransaction.java:130)
{quote}
{quote}
java.io.IOException: Cannot append; log is closed
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1146)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.appendNoSync(FSHLog.java:1120)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:266)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1211)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1158)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1514)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:478)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11671) TestEndToEndSplitTransaction fails on master
[
https://issues.apache.org/jira/browse/HBASE-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-11671:
Attachment: HBASE-11671-v2.patch
Ah, seems to be the problem was the the tests used to mimic the codepath of
SplitTransaction, but became obsolete after SplitTransaction started using
different codepath for zk-less assignment around postOpenDeployTasks().
Attaching better patch (in sense with it test passes with either useZK true or
false).
[~jxiang] could you take a look?
TestEndToEndSplitTransaction fails on master
Key: HBASE-11671
URL: https://issues.apache.org/jira/browse/HBASE-11671
Project: HBase
Issue Type: Bug
Components: Region Assignment, test
Affects Versions: 0.99.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 0.99.0
Attachments: HBASE-11671-v2.patch, HBASE-11671.patch
#testMasterOpsWhileSplitting() fails, seems after enabling ZK-less assignment.
Here're the only log snippets which seem relevant.
{quote}
java.io.IOException: Failed to report opened region to master:
TestSplit,lll,1407218018712.030c51654a5cbe9e0489a50762bb9075.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1731)
at
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.testMasterOpsWhileSplitting(TestEndToEndSplitTransaction.java:130)
{quote}
{quote}
java.io.IOException: Cannot append; log is closed
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1146)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.appendNoSync(FSHLog.java:1120)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:266)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1211)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1158)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1514)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:478)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11671) TestEndToEndSplitTransaction fails on master
[
https://issues.apache.org/jira/browse/HBASE-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-11671:
Status: Patch Available (was: Open)
TestEndToEndSplitTransaction fails on master
Key: HBASE-11671
URL: https://issues.apache.org/jira/browse/HBASE-11671
Project: HBase
Issue Type: Bug
Components: Region Assignment, test
Affects Versions: 0.99.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 0.99.0
Attachments: HBASE-11671-v2.patch, HBASE-11671.patch
#testMasterOpsWhileSplitting() fails, seems after enabling ZK-less assignment.
Here're the only log snippets which seem relevant.
{quote}
java.io.IOException: Failed to report opened region to master:
TestSplit,lll,1407218018712.030c51654a5cbe9e0489a50762bb9075.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1731)
at
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.testMasterOpsWhileSplitting(TestEndToEndSplitTransaction.java:130)
{quote}
{quote}
java.io.IOException: Cannot append; log is closed
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1146)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.appendNoSync(FSHLog.java:1120)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:266)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1211)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1158)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1514)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:478)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11673) TestIOFencing#testFencingAroundCompactionAfterWALSync fails
Qiang Tian created HBASE-11673:
--
Summary: TestIOFencing#testFencingAroundCompactionAfterWALSync
fails
Key: HBASE-11673
URL: https://issues.apache.org/jira/browse/HBASE-11673
Project: HBase
Issue Type: Bug
Reporter: Qiang Tian
got several test failure on the latest build:
{quote}
[tianq@bdvm101 surefire-reports]$ ls -1t|grep Tests run * |grep FAILURE
org.apache.hadoop.hbase.client.TestReplicasClient.txt:Tests run: 1, Failures:
0, Errors: 1, Skipped: 0, Time elapsed: 38.706 sec FAILURE!
org.apache.hadoop.hbase.master.TestMasterOperationsForRegionReplicas.txt:Tests
run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 30.669 sec
FAILURE!
org.apache.hadoop.hbase.regionserver.TestRegionReplicas.txt:Tests run: 1,
Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 39.113 sec FAILURE!
org.apache.hadoop.hbase.TestIOFencing.txt:Tests run: 2, Failures: 1, Errors: 0,
Skipped: 0, Time elapsed: 177.071 sec FAILURE!
{quote}
the first one:
{quote}
failure message=Timed out waiting for the region to flush
type=java.lang.AssertionErrorjava.lang.AssertionError: Timed out waiting for
the region to flush
-at org.junit.Assert.fail(Assert.java:88)
-at org.junit.Assert.assertTrue(Assert.java:41)
-at org.apache.hadoop.hbase.TestIOFencing.doTest(TestIOFencing.java:291)
-at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompactionAfterWALSync(TestIOFencing.java:236)
-at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
-at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
-at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
-at java.lang.reflect.Method.invoke(Method.java:606)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11589) AccessControlException handling in HBase rpc server and client. AccessControlException should be a not retriable exception
[
https://issues.apache.org/jira/browse/HBASE-11589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang Tian updated HBASE-11589:
---
Status: Patch Available (was: Open)
AccessControlException handling in HBase rpc server and client.
AccessControlException should be a not retriable exception
--
Key: HBASE-11589
URL: https://issues.apache.org/jira/browse/HBASE-11589
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.98.3
Environment: SLES 11 SP1
Reporter: Kashif J S
Assignee: Qiang Tian
Attachments: hbase-11589-master-v1.patch, hbase-11589-master.patch
RPC server does not handle the AccessControlException thrown by
authorizeConnection failure properly and in return sends IOException to the
HBase client.
Ultimately the client does retries and gets RetriesExhaustedException but
does not getting any link or information or stack trace about
AccessControlException.
In short summary, upon inspection of RPCServer.java, it seems
for the Listener, the Reader read code as below does not handle
AccessControlException
{noformat}
void doRead(….
…..
…..
try {
count = c.readAndProcess(); // This readAndProcess method throws
AccessControlException from processOneRpc(byte[] buf) which is not handled ?
} catch (InterruptedException ieo) {
throw ieo;
} catch (Exception e) {
LOG.warn(getName() + : count of bytes read: + count, e);
count = -1; //so that the (count 0) block is executed
}
{noformat}
Below is the client logs if authorizeConnection throws AccessControlException:
2014-07-24 19:40:58,768 INFO [main]
client.HConnectionManager$HConnectionImplementation: getMaster attempt 7 of 7
failed; no more retrying.
com.google.protobuf.ServiceException: java.io.IOException: Call to
host-10-18-40-101/10.18.40.101:6 failed on local exception:
java.io.EOFException
at
org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674)
at
org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885)
at
org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3302)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3329)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:605)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:496)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:430)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:311)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:59)
at
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.IfNode.interpret(IfNode.java:117)
[jira] [Commented] (HBASE-11642) EOL 0.96
[ https://issues.apache.org/jira/browse/HBASE-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085992#comment-14085992 ] Ishan Chhabra commented on HBASE-11642: --- Hmm, I agree and can relate to the pain associated with a co-ordinated upgrade having gone through it myself. Thinking about it more, I agree a full EOL is not a good idea for 0.94, but is there are way to make its status more clearer and explicit (maybe extended maintenance like python 2.x) so that people running smaller clusters would consider upgrades and newer users don't use 0.94 accidentally (I see some of them doing that mostly because they start with CDH 4). It is easy for a casual observer to think that 0.94 is having active releases, whereas it is made clear in the JIRAs that new features should not be added to this branch. As a side note, this line should be definitely fixed in the download pages (eg. http://www.carfab.com/apachesoftware/hbase/) The 0.96.x series supercedes 0.94.x. We are leaving the 'stable' pointer on the latest 0.94.x for now while 0.96 is still 'fresh'. EOL 0.96 Key: HBASE-11642 URL: https://issues.apache.org/jira/browse/HBASE-11642 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Do the work to EOL 0.96. + No more patches on 0.96 + Remove 0.96 from downloads. + If user has issue with 0.96 and needs fix, fix it in 0.98 and have the user upgrade to get the fix. + Write email to user list stating 0.96 has been EOL'd September 1st? And add notice to refguide. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11671) TestEndToEndSplitTransaction fails on master
[
https://issues.apache.org/jira/browse/HBASE-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086002#comment-14086002
]
Hadoop QA commented on HBASE-11671:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12659856/HBASE-11671-v2.patch
against trunk revision .
ATTACHMENT ID: 12659856
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestClockSkewDetection
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.ipc.TestIPC
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10294//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10294//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10294//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10294//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10294//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10294//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10294//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10294//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10294//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10294//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10294//console
This message is automatically generated.
TestEndToEndSplitTransaction fails on master
Key: HBASE-11671
URL: https://issues.apache.org/jira/browse/HBASE-11671
Project: HBase
Issue Type: Bug
Components: Region Assignment, test
Affects Versions: 0.99.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 0.99.0
Attachments: HBASE-11671-v2.patch, HBASE-11671.patch
#testMasterOpsWhileSplitting() fails, seems after enabling ZK-less assignment.
Here're the only log snippets which seem relevant.
{quote}
java.io.IOException: Failed to report opened region to master:
TestSplit,lll,1407218018712.030c51654a5cbe9e0489a50762bb9075.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1731)
at
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.testMasterOpsWhileSplitting(TestEndToEndSplitTransaction.java:130)
{quote}
{quote}
java.io.IOException: Cannot append; log is closed
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1146)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.appendNoSync(FSHLog.java:1120)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:266)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1211)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1158)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1514)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:478)
at
[jira] [Commented] (HBASE-11447) Proposal for a generic transaction API for HBase
[ https://issues.apache.org/jira/browse/HBASE-11447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086006#comment-14086006 ] John de Roo commented on HBASE-11447: - I like the idea of providing the option to allow non-transactional behavior for the TransactionTable. I'll add it to the 0.6 version. Proposal for a generic transaction API for HBase Key: HBASE-11447 URL: https://issues.apache.org/jira/browse/HBASE-11447 Project: HBase Issue Type: New Feature Components: Client Affects Versions: 1.0.0 Environment: Any. Reporter: John de Roo Priority: Minor Labels: features, newbie Fix For: 1.0.0 Attachments: Proposal for a common transactional API for HBase v0.3_1.pdf, Proposal for a common transactional API for HBase v0.4_1.pdf, Proposal for a common transactional API for HBase v0.5.pdf, Re Proposal for a generic transaction API for HBase.htm HBase transaction management today is provided by a number of products, each implementing a different API, each having different strengths. The lack of a common API for transactional interfaces means that applications need to be coded to work with a specific Transaction Manager. This proposal outlines an API which, if implemented by the different Transaction Manager vendors would provide stability and choice to HBase application developers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11447) Proposal for a generic transaction API for HBase
[ https://issues.apache.org/jira/browse/HBASE-11447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086017#comment-14086017 ] John de Roo commented on HBASE-11447: - Forgot to mention that I'll add in a snapshot isolation option. Proposal for a generic transaction API for HBase Key: HBASE-11447 URL: https://issues.apache.org/jira/browse/HBASE-11447 Project: HBase Issue Type: New Feature Components: Client Affects Versions: 1.0.0 Environment: Any. Reporter: John de Roo Priority: Minor Labels: features, newbie Fix For: 1.0.0 Attachments: Proposal for a common transactional API for HBase v0.3_1.pdf, Proposal for a common transactional API for HBase v0.4_1.pdf, Proposal for a common transactional API for HBase v0.5.pdf, Re Proposal for a generic transaction API for HBase.htm HBase transaction management today is provided by a number of products, each implementing a different API, each having different strengths. The lack of a common API for transactional interfaces means that applications need to be coded to work with a specific Transaction Manager. This proposal outlines an API which, if implemented by the different Transaction Manager vendors would provide stability and choice to HBase application developers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11447) Proposal for a generic transaction API for HBase
[ https://issues.apache.org/jira/browse/HBASE-11447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086015#comment-14086015 ] John de Roo commented on HBASE-11447: - I agree, thanks James. The proposal must work for Continuuity and not be too difficult to implement. Gary, I'm attending a meeting with Continuuity tomorrow. Not sure whether you're involved. Probably not the right forum to discuss the details, could we set up a time to discuss the differences as James suggests? I will fit around your schedule, but would prefer your afternoons (NZ is 19 hours ahead). Proposal for a generic transaction API for HBase Key: HBASE-11447 URL: https://issues.apache.org/jira/browse/HBASE-11447 Project: HBase Issue Type: New Feature Components: Client Affects Versions: 1.0.0 Environment: Any. Reporter: John de Roo Priority: Minor Labels: features, newbie Fix For: 1.0.0 Attachments: Proposal for a common transactional API for HBase v0.3_1.pdf, Proposal for a common transactional API for HBase v0.4_1.pdf, Proposal for a common transactional API for HBase v0.5.pdf, Re Proposal for a generic transaction API for HBase.htm HBase transaction management today is provided by a number of products, each implementing a different API, each having different strengths. The lack of a common API for transactional interfaces means that applications need to be coded to work with a specific Transaction Manager. This proposal outlines an API which, if implemented by the different Transaction Manager vendors would provide stability and choice to HBase application developers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11589) AccessControlException handling in HBase rpc server and client. AccessControlException should be a not retriable exception
[
https://issues.apache.org/jira/browse/HBASE-11589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086019#comment-14086019
]
Hadoop QA commented on HBASE-11589:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12659807/hbase-11589-master-v1.patch
against trunk revision .
ATTACHMENT ID: 12659807
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.ipc.TestIPC
org.apache.hadoop.hbase.master.TestClockSkewDetection
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10295//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10295//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10295//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10295//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10295//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10295//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10295//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10295//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10295//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10295//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10295//console
This message is automatically generated.
AccessControlException handling in HBase rpc server and client.
AccessControlException should be a not retriable exception
--
Key: HBASE-11589
URL: https://issues.apache.org/jira/browse/HBASE-11589
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.98.3
Environment: SLES 11 SP1
Reporter: Kashif J S
Assignee: Qiang Tian
Attachments: hbase-11589-master-v1.patch, hbase-11589-master.patch
RPC server does not handle the AccessControlException thrown by
authorizeConnection failure properly and in return sends IOException to the
HBase client.
Ultimately the client does retries and gets RetriesExhaustedException but
does not getting any link or information or stack trace about
AccessControlException.
In short summary, upon inspection of RPCServer.java, it seems
for the Listener, the Reader read code as below does not handle
AccessControlException
{noformat}
void doRead(….
…..
…..
try {
count = c.readAndProcess(); // This readAndProcess method throws
AccessControlException from processOneRpc(byte[] buf) which is not handled ?
} catch (InterruptedException ieo) {
throw ieo;
} catch (Exception e) {
LOG.warn(getName() + : count of bytes read: + count, e);
count = -1; //so that the (count 0) block is executed
}
{noformat}
[jira] [Commented] (HBASE-11662) Launching shell with long-form --debug fails
[
https://issues.apache.org/jira/browse/HBASE-11662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086024#comment-14086024
]
Hudson commented on HBASE-11662:
FAILURE: Integrated in HBase-1.0 #85 (See
[https://builds.apache.org/job/HBase-1.0/85/])
HBASE-11662 shell should properly handle long form --debug. (apurtell: rev
6a7f923c0d4ca2942b7bdcd32ca0ef8c3293902b)
* bin/hirb.rb
Launching shell with long-form --debug fails
Key: HBASE-11662
URL: https://issues.apache.org/jira/browse/HBASE-11662
Project: HBase
Issue Type: Bug
Components: shell
Reporter: Sean Busbey
Assignee: Sean Busbey
Fix For: 0.99.0, 0.98.5, 2.0.0
Attachments: HBASE_11662-v1.patch
Even though the help says both '-d' and '--debug' are allowed, the shell
fails if you use the long form.
{noformat}
busbey2-MBA:hbase busbey$ bin/hbase shell --help
Usage: shell [OPTIONS] [SCRIPTFILE [ARGUMENTS]]
--format=OPTIONFormatter for outputting results.
Valid options are: console, html.
(Default: console)
-d | --debug Set DEBUG log levels.
-h | --helpThis help.
busbey2-MBA:hbase busbey$ bin/hbase shell --debug
Setting DEBUG log level...
HBase Shell; enter 'helpRETURN' for list of supported commands.
Type exitRETURN to leave the HBase Shell
Version 2.0.0-SNAPSHOT, r4d005b70a0fda4be5a8ead84ff5f54fceb3c637a, Mon Aug 4
14:17:22 CDT 2014
IRB::UnrecognizedSwitch: Unrecognized switch: --debug
Raise at
file:/Users/busbey/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/META-INF/jruby.home/lib/ruby/1.8/e2mmap.rb:167
fail at
file:/Users/busbey/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/META-INF/jruby.home/lib/ruby/1.8/e2mmap.rb:95
parse_opts at
file:/Users/busbey/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/META-INF/jruby.home/lib/ruby/1.8/irb/init.rb:194
setup at
file:/Users/busbey/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/META-INF/jruby.home/lib/ruby/1.8/irb/init.rb:19
start at /Users/busbey/projects/hbase/bin/../bin/hirb.rb:170
(root) at /Users/busbey/projects/hbase/bin/../bin/hirb.rb:190
busbey2-MBA:hbase busbey$ bin/hbase shell -d
Setting DEBUG log level...
HBase Shell; enter 'helpRETURN' for list of supported commands.
Type exitRETURN to leave the HBase Shell
Version 2.0.0-SNAPSHOT, r4d005b70a0fda4be5a8ead84ff5f54fceb3c637a, Mon Aug 4
14:17:22 CDT 2014
jruby-1.7.3 :001 quit
busbey2-MBA:hbase busbey$
{noformat}
The problem is that we're sending the debug flag through to the IRB
initialization as is, and it only recognizes the '-d' form.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11668) Re-add HBASE_LIBRARY_PATH to bin/hbase
[
https://issues.apache.org/jira/browse/HBASE-11668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086025#comment-14086025
]
Hudson commented on HBASE-11668:
FAILURE: Integrated in HBase-1.0 #85 (See
[https://builds.apache.org/job/HBase-1.0/85/])
HBASE-11668 Re-add HBASE_LIBRARY_PATH to bin/hbase (Esteban Gutierrez)
(apurtell: rev bd246347077b162f0e2888e75c517dd47587b35d)
* bin/hbase
Re-add HBASE_LIBRARY_PATH to bin/hbase
--
Key: HBASE-11668
URL: https://issues.apache.org/jira/browse/HBASE-11668
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 0.96.2, 1.0.0, 0.94.21, 0.98.4, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Trivial
Fix For: 0.99.0, 0.98.5, 2.0.0
Attachments: HBASE-11668.patch, HBASE-11668.patch
This is a follow up of HBASE-3533. {{HBASE_LIBRARY_PATH}} is the suggested
environment variable in the documentation to specify the location of the
native libraries for HBase, however it is never used.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11589) AccessControlException handling in HBase rpc server and client. AccessControlException should be a not retriable exception
[
https://issues.apache.org/jira/browse/HBASE-11589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086028#comment-14086028
]
Qiang Tian commented on HBASE-11589:
all test failures are about java.net.UnknownHostException:
asf901.ygridcore.net: asf901.ygridcore.net, looks temporary DNS failure?
AccessControlException handling in HBase rpc server and client.
AccessControlException should be a not retriable exception
--
Key: HBASE-11589
URL: https://issues.apache.org/jira/browse/HBASE-11589
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.98.3
Environment: SLES 11 SP1
Reporter: Kashif J S
Assignee: Qiang Tian
Attachments: hbase-11589-master-v1.patch, hbase-11589-master.patch
RPC server does not handle the AccessControlException thrown by
authorizeConnection failure properly and in return sends IOException to the
HBase client.
Ultimately the client does retries and gets RetriesExhaustedException but
does not getting any link or information or stack trace about
AccessControlException.
In short summary, upon inspection of RPCServer.java, it seems
for the Listener, the Reader read code as below does not handle
AccessControlException
{noformat}
void doRead(….
…..
…..
try {
count = c.readAndProcess(); // This readAndProcess method throws
AccessControlException from processOneRpc(byte[] buf) which is not handled ?
} catch (InterruptedException ieo) {
throw ieo;
} catch (Exception e) {
LOG.warn(getName() + : count of bytes read: + count, e);
count = -1; //so that the (count 0) block is executed
}
{noformat}
Below is the client logs if authorizeConnection throws AccessControlException:
2014-07-24 19:40:58,768 INFO [main]
client.HConnectionManager$HConnectionImplementation: getMaster attempt 7 of 7
failed; no more retrying.
com.google.protobuf.ServiceException: java.io.IOException: Call to
host-10-18-40-101/10.18.40.101:6 failed on local exception:
java.io.EOFException
at
org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674)
at
org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885)
at
org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3302)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3329)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:605)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:496)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:430)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:311)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:59)
at
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at
[jira] [Commented] (HBASE-11673) TestIOFencing#testFencingAroundCompactionAfterWALSync fails
[
https://issues.apache.org/jira/browse/HBASE-11673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086030#comment-14086030
]
Qiang Tian commented on HBASE-11673:
I can repro consistently. but the hadoop QA run looks good
TestIOFencing#testFencingAroundCompactionAfterWALSync fails
---
Key: HBASE-11673
URL: https://issues.apache.org/jira/browse/HBASE-11673
Project: HBase
Issue Type: Bug
Reporter: Qiang Tian
got several test failure on the latest build:
{quote}
[tianq@bdvm101 surefire-reports]$ ls -1t|grep Tests run * |grep
FAILURE
org.apache.hadoop.hbase.client.TestReplicasClient.txt:Tests run: 1, Failures:
0, Errors: 1, Skipped: 0, Time elapsed: 38.706 sec FAILURE!
org.apache.hadoop.hbase.master.TestMasterOperationsForRegionReplicas.txt:Tests
run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 30.669 sec
FAILURE!
org.apache.hadoop.hbase.regionserver.TestRegionReplicas.txt:Tests run: 1,
Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 39.113 sec FAILURE!
org.apache.hadoop.hbase.TestIOFencing.txt:Tests run: 2, Failures: 1, Errors:
0, Skipped: 0, Time elapsed: 177.071 sec FAILURE!
{quote}
the first one:
{quote}
failure message=Timed out waiting for the region to flush
type=java.lang.AssertionErrorjava.lang.AssertionError: Timed out waiting
for the region to flush
-at org.junit.Assert.fail(Assert.java:88)
-at org.junit.Assert.assertTrue(Assert.java:41)
-at org.apache.hadoop.hbase.TestIOFencing.doTest(TestIOFencing.java:291)
-at
org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompactionAfterWALSync(TestIOFencing.java:236)
-at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
-at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
-at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
-at java.lang.reflect.Method.invoke(Method.java:606)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11674) LoadIncrementalHFiles should be more verbose after unrecoverable error
Jan Lukavsky created HBASE-11674: Summary: LoadIncrementalHFiles should be more verbose after unrecoverable error Key: HBASE-11674 URL: https://issues.apache.org/jira/browse/HBASE-11674 Project: HBase Issue Type: Improvement Components: mapreduce Affects Versions: 0.98.5 Reporter: Jan Lukavsky Assignee: Jan Lukavsky LoadIncrementalHFiles should give more information after failure to load data to regionserver. Currently, it logs only Encountered unrecoverable error from region server, but doesn't give information about * which region server it talked to * which was the region that failed to load data In order to help understand what is going on, the log should contain both these information. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11674) LoadIncrementalHFiles should be more verbose after unrecoverable error
[ https://issues.apache.org/jira/browse/HBASE-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lukavsky updated HBASE-11674: - Status: Patch Available (was: Open) I did not find out, how to get all the information (RegionServerCallable#getLocation is protected), but I suppose that the following patch could do the work. LoadIncrementalHFiles should be more verbose after unrecoverable error -- Key: HBASE-11674 URL: https://issues.apache.org/jira/browse/HBASE-11674 Project: HBase Issue Type: Improvement Components: mapreduce Affects Versions: 0.98.5 Reporter: Jan Lukavsky Assignee: Jan Lukavsky Attachments: HBASE-11674.patch LoadIncrementalHFiles should give more information after failure to load data to regionserver. Currently, it logs only Encountered unrecoverable error from region server, but doesn't give information about * which region server it talked to * which was the region that failed to load data In order to help understand what is going on, the log should contain both these information. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11674) LoadIncrementalHFiles should be more verbose after unrecoverable error
[ https://issues.apache.org/jira/browse/HBASE-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lukavsky updated HBASE-11674: - Attachment: HBASE-11674.patch LoadIncrementalHFiles should be more verbose after unrecoverable error -- Key: HBASE-11674 URL: https://issues.apache.org/jira/browse/HBASE-11674 Project: HBase Issue Type: Improvement Components: mapreduce Affects Versions: 0.98.5 Reporter: Jan Lukavsky Assignee: Jan Lukavsky Attachments: HBASE-11674.patch LoadIncrementalHFiles should give more information after failure to load data to regionserver. Currently, it logs only Encountered unrecoverable error from region server, but doesn't give information about * which region server it talked to * which was the region that failed to load data In order to help understand what is going on, the log should contain both these information. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11675) Major compactions change query results
chendihao created HBASE-11675: - Summary: Major compactions change query results Key: HBASE-11675 URL: https://issues.apache.org/jira/browse/HBASE-11675 Project: HBase Issue Type: Bug Reporter: chendihao The bug mentioned in http://hbase.apache.org/book.html 5.9.2.2. Major compactions change query results “...create three cell versions at t1, t2 and t3, with a maximum-versions setting of 2. So when getting all versions, only the values at t2 and t3 will be returned. But if you delete the version at t2 or t3, the one at t1 will appear again. Obviously, once a major compaction has run, such behavior will not be the case anymore...” -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11674) LoadIncrementalHFiles should be more verbose after unrecoverable error
[
https://issues.apache.org/jira/browse/HBASE-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086101#comment-14086101
]
Hadoop QA commented on HBASE-11674:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12659870/HBASE-11674.patch
against trunk revision .
ATTACHMENT ID: 12659870
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestClockSkewDetection
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.ipc.TestIPC
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10296//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10296//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10296//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10296//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10296//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10296//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10296//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10296//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10296//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10296//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10296//console
This message is automatically generated.
LoadIncrementalHFiles should be more verbose after unrecoverable error
--
Key: HBASE-11674
URL: https://issues.apache.org/jira/browse/HBASE-11674
Project: HBase
Issue Type: Improvement
Components: mapreduce
Affects Versions: 0.98.5
Reporter: Jan Lukavsky
Assignee: Jan Lukavsky
Attachments: HBASE-11674.patch
LoadIncrementalHFiles should give more information after failure to load data
to regionserver. Currently, it logs only Encountered unrecoverable error
from region server, but doesn't give information about
* which region server it talked to
* which was the region that failed to load data
In order to help understand what is going on, the log should contain both
these information.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11674) LoadIncrementalHFiles should be more verbose after unrecoverable error
[ https://issues.apache.org/jira/browse/HBASE-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Lukavsky updated HBASE-11674: - Attachment: HBASE-11674-ii.patch I accidentally removed logging of the exception, fixing that. LoadIncrementalHFiles should be more verbose after unrecoverable error -- Key: HBASE-11674 URL: https://issues.apache.org/jira/browse/HBASE-11674 Project: HBase Issue Type: Improvement Components: mapreduce Affects Versions: 0.98.5 Reporter: Jan Lukavsky Assignee: Jan Lukavsky Attachments: HBASE-11674-ii.patch, HBASE-11674.patch LoadIncrementalHFiles should give more information after failure to load data to regionserver. Currently, it logs only Encountered unrecoverable error from region server, but doesn't give information about * which region server it talked to * which was the region that failed to load data In order to help understand what is going on, the log should contain both these information. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11676) Scan FORMATTER is not applied for columns using non-printable name in shell
t samkawa created HBASE-11676: - Summary: Scan FORMATTER is not applied for columns using non-printable name in shell Key: HBASE-11676 URL: https://issues.apache.org/jira/browse/HBASE-11676 Project: HBase Issue Type: Bug Components: shell Environment: 0.98 + hadoop2.2.0 Reporter: t samkawa Priority: Minor The FORMATTER does not work if the target cell`s column uses binary name. hbase create test1, f hbase incr test1, row1 , f:a, 1 hbase incr test1, row1 , f:\x11, 1 hbase scan test1, COLUMNS=[f:\x11:toLong,f:a:toLong] ROW COLUMN+CELL row1column=f:\x11, ..., value=\x00\x00\x00\x00\x00\x00\x00\x01 row1column=f:a, ..., value=1 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11674) LoadIncrementalHFiles should be more verbose after unrecoverable error
[
https://issues.apache.org/jira/browse/HBASE-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086127#comment-14086127
]
Hadoop QA commented on HBASE-11674:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12659875/HBASE-11674-ii.patch
against trunk revision .
ATTACHMENT ID: 12659875
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.ipc.TestIPC
org.apache.hadoop.hbase.master.TestClockSkewDetection
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10297//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10297//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10297//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10297//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10297//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10297//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10297//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10297//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10297//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10297//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10297//console
This message is automatically generated.
LoadIncrementalHFiles should be more verbose after unrecoverable error
--
Key: HBASE-11674
URL: https://issues.apache.org/jira/browse/HBASE-11674
Project: HBase
Issue Type: Improvement
Components: mapreduce
Affects Versions: 0.98.5
Reporter: Jan Lukavsky
Assignee: Jan Lukavsky
Attachments: HBASE-11674-ii.patch, HBASE-11674.patch
LoadIncrementalHFiles should give more information after failure to load data
to regionserver. Currently, it logs only Encountered unrecoverable error
from region server, but doesn't give information about
* which region server it talked to
* which was the region that failed to load data
In order to help understand what is going on, the log should contain both
these information.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11662) Launching shell with long-form --debug fails
[
https://issues.apache.org/jira/browse/HBASE-11662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086135#comment-14086135
]
Hudson commented on HBASE-11662:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #411 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/411/])
HBASE-11662 shell should properly handle long form --debug. (apurtell: rev
74c7169252e67ff2f578a0d61b8cb11276ddfcbe)
* bin/hirb.rb
Launching shell with long-form --debug fails
Key: HBASE-11662
URL: https://issues.apache.org/jira/browse/HBASE-11662
Project: HBase
Issue Type: Bug
Components: shell
Reporter: Sean Busbey
Assignee: Sean Busbey
Fix For: 0.99.0, 0.98.5, 2.0.0
Attachments: HBASE_11662-v1.patch
Even though the help says both '-d' and '--debug' are allowed, the shell
fails if you use the long form.
{noformat}
busbey2-MBA:hbase busbey$ bin/hbase shell --help
Usage: shell [OPTIONS] [SCRIPTFILE [ARGUMENTS]]
--format=OPTIONFormatter for outputting results.
Valid options are: console, html.
(Default: console)
-d | --debug Set DEBUG log levels.
-h | --helpThis help.
busbey2-MBA:hbase busbey$ bin/hbase shell --debug
Setting DEBUG log level...
HBase Shell; enter 'helpRETURN' for list of supported commands.
Type exitRETURN to leave the HBase Shell
Version 2.0.0-SNAPSHOT, r4d005b70a0fda4be5a8ead84ff5f54fceb3c637a, Mon Aug 4
14:17:22 CDT 2014
IRB::UnrecognizedSwitch: Unrecognized switch: --debug
Raise at
file:/Users/busbey/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/META-INF/jruby.home/lib/ruby/1.8/e2mmap.rb:167
fail at
file:/Users/busbey/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/META-INF/jruby.home/lib/ruby/1.8/e2mmap.rb:95
parse_opts at
file:/Users/busbey/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/META-INF/jruby.home/lib/ruby/1.8/irb/init.rb:194
setup at
file:/Users/busbey/.m2/repository/org/jruby/jruby-complete/1.6.8/jruby-complete-1.6.8.jar!/META-INF/jruby.home/lib/ruby/1.8/irb/init.rb:19
start at /Users/busbey/projects/hbase/bin/../bin/hirb.rb:170
(root) at /Users/busbey/projects/hbase/bin/../bin/hirb.rb:190
busbey2-MBA:hbase busbey$ bin/hbase shell -d
Setting DEBUG log level...
HBase Shell; enter 'helpRETURN' for list of supported commands.
Type exitRETURN to leave the HBase Shell
Version 2.0.0-SNAPSHOT, r4d005b70a0fda4be5a8ead84ff5f54fceb3c637a, Mon Aug 4
14:17:22 CDT 2014
jruby-1.7.3 :001 quit
busbey2-MBA:hbase busbey$
{noformat}
The problem is that we're sending the debug flag through to the IRB
initialization as is, and it only recognizes the '-d' form.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11668) Re-add HBASE_LIBRARY_PATH to bin/hbase
[
https://issues.apache.org/jira/browse/HBASE-11668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086136#comment-14086136
]
Hudson commented on HBASE-11668:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #411 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/411/])
HBASE-11668 Re-add HBASE_LIBRARY_PATH to bin/hbase (Esteban Gutierrez)
(apurtell: rev f10effb796b6ce72a87bf59db1dd0ddf10c366e6)
* bin/hbase
Re-add HBASE_LIBRARY_PATH to bin/hbase
--
Key: HBASE-11668
URL: https://issues.apache.org/jira/browse/HBASE-11668
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 0.96.2, 1.0.0, 0.94.21, 0.98.4, 2.0.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Trivial
Fix For: 0.99.0, 0.98.5, 2.0.0
Attachments: HBASE-11668.patch, HBASE-11668.patch
This is a follow up of HBASE-3533. {{HBASE_LIBRARY_PATH}} is the suggested
environment variable in the documentation to specify the location of the
native libraries for HBase, however it is never used.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11675) Major compactions change query results
[ https://issues.apache.org/jira/browse/HBASE-11675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086161#comment-14086161 ] Jean-Marc Spaggiari commented on HBASE-11675: - Hi [~tobe], are you going to propose a patch on that? Based on previous discussions on the mailing list, I think conclusion has been that this is consider as normal behavior. Can you please comment? Thanks. Major compactions change query results -- Key: HBASE-11675 URL: https://issues.apache.org/jira/browse/HBASE-11675 Project: HBase Issue Type: Bug Reporter: chendihao The bug mentioned in http://hbase.apache.org/book.html 5.9.2.2. Major compactions change query results “...create three cell versions at t1, t2 and t3, with a maximum-versions setting of 2. So when getting all versions, only the values at t2 and t3 will be returned. But if you delete the version at t2 or t3, the one at t1 will appear again. Obviously, once a major compaction has run, such behavior will not be the case anymore...” -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11675) Major compactions change query results
[ https://issues.apache.org/jira/browse/HBASE-11675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086170#comment-14086170 ] chendihao commented on HBASE-11675: --- [~jmspaggi] We're working on it. At present, there's no perfect solution but I don't think it's a normal behavior for our users. We will make a patch when we work it out. Major compactions change query results -- Key: HBASE-11675 URL: https://issues.apache.org/jira/browse/HBASE-11675 Project: HBase Issue Type: Bug Reporter: chendihao The bug mentioned in http://hbase.apache.org/book.html 5.9.2.2. Major compactions change query results “...create three cell versions at t1, t2 and t3, with a maximum-versions setting of 2. So when getting all versions, only the values at t2 and t3 will be returned. But if you delete the version at t2 or t3, the one at t1 will appear again. Obviously, once a major compaction has run, such behavior will not be the case anymore...” -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11677) Make Logger instance modifiers consistent
Sean Busbey created HBASE-11677:
---
Summary: Make Logger instance modifiers consistent
Key: HBASE-11677
URL: https://issues.apache.org/jira/browse/HBASE-11677
Project: HBase
Issue Type: Task
Reporter: Sean Busbey
Priority: Minor
We have some instances of Logger that are missing one of being private, static,
and final.
ex from HealthChecker.java, missing final
{code}
private static Log LOG = LogFactory.getLog(HealthChecker.class);
{code}
* Clean up where possible by making {{private static final}}
* If we can't, add a non-javadoc note about why
One way to look for problematic instances is to grep for initial assignment for
the commonly used LOG member, e.g.
* missing final: {{grep -r LOG = * | grep -v final}}
* missing static: {{grep -r LOG = * | grep -v static}}
* missing private: {{grep -r LOG = * | grep -v private}}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11666) Enforce JDK7 javac for builds on branch-1 and master
[ https://issues.apache.org/jira/browse/HBASE-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086283#comment-14086283 ] Jonathan Hsieh commented on HBASE-11666: +1 Enforce JDK7 javac for builds on branch-1 and master Key: HBASE-11666 URL: https://issues.apache.org/jira/browse/HBASE-11666 Project: HBase Issue Type: Task Components: build, documentation Affects Versions: 1.0.0, 2.0.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Attachments: HBASE_11666-v1.patch, HBASE_11666-v2.patch Per [the discussion on JDK versions|http://search-hadoop.com/m/DHED4Zlz0R1], we require Java 7 for 1.0+ (also we have code that only compiles under Java 7). Right now, if you attempt to build with JDK6, Maven will happily oblige and then give you a cannot find symbol error about a Java 7 class we use. We should * update the Build HBase guide to reference our supported Java versions docs * update our pom so that Maven will correctly complain about jdk versions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11666) Enforce JDK7 javac for builds on branch-1 and master
[ https://issues.apache.org/jira/browse/HBASE-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11666: Attachment: HBASE_11666-v3.patch It occurred to me to do search of the rest of the codebase, which turned up a couple of comments and an error message where we should assert Java 7 instead of Java 6. Patch updated; last time barring feedback. Enforce JDK7 javac for builds on branch-1 and master Key: HBASE-11666 URL: https://issues.apache.org/jira/browse/HBASE-11666 Project: HBase Issue Type: Task Components: build, documentation Affects Versions: 1.0.0, 2.0.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Attachments: HBASE_11666-v1.patch, HBASE_11666-v2.patch, HBASE_11666-v3.patch Per [the discussion on JDK versions|http://search-hadoop.com/m/DHED4Zlz0R1], we require Java 7 for 1.0+ (also we have code that only compiles under Java 7). Right now, if you attempt to build with JDK6, Maven will happily oblige and then give you a cannot find symbol error about a Java 7 class we use. We should * update the Build HBase guide to reference our supported Java versions docs * update our pom so that Maven will correctly complain about jdk versions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11666) Enforce JDK7 javac for builds on branch-1 and master
[ https://issues.apache.org/jira/browse/HBASE-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11666: Status: Patch Available (was: In Progress) Enforce JDK7 javac for builds on branch-1 and master Key: HBASE-11666 URL: https://issues.apache.org/jira/browse/HBASE-11666 Project: HBase Issue Type: Task Components: build, documentation Affects Versions: 1.0.0, 2.0.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Attachments: HBASE_11666-v1.patch, HBASE_11666-v2.patch, HBASE_11666-v3.patch Per [the discussion on JDK versions|http://search-hadoop.com/m/DHED4Zlz0R1], we require Java 7 for 1.0+ (also we have code that only compiles under Java 7). Right now, if you attempt to build with JDK6, Maven will happily oblige and then give you a cannot find symbol error about a Java 7 class we use. We should * update the Build HBase guide to reference our supported Java versions docs * update our pom so that Maven will correctly complain about jdk versions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11666) Enforce JDK7 javac for builds on branch-1 and master
[ https://issues.apache.org/jira/browse/HBASE-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-11666: Status: In Progress (was: Patch Available) Enforce JDK7 javac for builds on branch-1 and master Key: HBASE-11666 URL: https://issues.apache.org/jira/browse/HBASE-11666 Project: HBase Issue Type: Task Components: build, documentation Affects Versions: 1.0.0, 2.0.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Attachments: HBASE_11666-v1.patch, HBASE_11666-v2.patch, HBASE_11666-v3.patch Per [the discussion on JDK versions|http://search-hadoop.com/m/DHED4Zlz0R1], we require Java 7 for 1.0+ (also we have code that only compiles under Java 7). Right now, if you attempt to build with JDK6, Maven will happily oblige and then give you a cannot find symbol error about a Java 7 class we use. We should * update the Build HBase guide to reference our supported Java versions docs * update our pom so that Maven will correctly complain about jdk versions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11672) Add IntegrationTestBigLinkedListWithRegionMovement
[ https://issues.apache.org/jira/browse/HBASE-11672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086297#comment-14086297 ] Jonathan Hsieh commented on HBASE-11672: Test looks fine but does it make sense to just use command line args to trigger this specific monkey instead of having a test specifically for it? Maybe add the mover monkey here [1] so you can do Something like : 'hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList --monkey xxx ...' ? [1] https://github.com/apache/hbase/blob/master/hbase-it/src/test/java/org/apache/hadoop/hbase/chaos/factories/MonkeyFactory.java Add IntegrationTestBigLinkedListWithRegionMovement -- Key: HBASE-11672 URL: https://issues.apache.org/jira/browse/HBASE-11672 Project: HBase Issue Type: Test Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Fix For: 0.99.0, 2.0.0, 0.98.6 Attachments: HBASE-11672.patch An integration test that extends ITBLL with a fixed monkey policy that moves a random region of the table very often. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11647) MOB integration testing
[ https://issues.apache.org/jira/browse/HBASE-11647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11647: --- Summary: MOB integration testing (was: Integration testings) MOB integration testing --- Key: HBASE-11647 URL: https://issues.apache.org/jira/browse/HBASE-11647 Project: HBase Issue Type: Sub-task Components: Performance, test Reporter: Jingcheng Du Assignee: Jingcheng Du The integration testings include the integration function testing and performance testing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11647) MOB integration testing
[ https://issues.apache.org/jira/browse/HBASE-11647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086319#comment-14086319 ] Jonathan Hsieh commented on HBASE-11647: This idelaly would be a set of tests derived from IntegrationTest that would constantly read and write MOB sized data to a cf with the mob feature enabled. Along with the regular chaos monkeys, we would also need actions that would trigger the ttl and sweep jobs and also alter mob thresholds and cause normal compactions. MOB integration testing --- Key: HBASE-11647 URL: https://issues.apache.org/jira/browse/HBASE-11647 Project: HBase Issue Type: Sub-task Components: Performance, test Reporter: Jingcheng Du Assignee: Jingcheng Du The integration testings include the integration function testing and performance testing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11645) Snapshot for MOB
[ https://issues.apache.org/jira/browse/HBASE-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086323#comment-14086323 ] Jonathan Hsieh commented on HBASE-11645: A good template to follow is the snapshot unit tests -- but in these cases add mob data in the target tables. Snapshot for MOB Key: HBASE-11645 URL: https://issues.apache.org/jira/browse/HBASE-11645 Project: HBase Issue Type: Sub-task Components: snapshots Reporter: Jingcheng Du Assignee: Jingcheng Du Add snapshot support for MOB. In the initial implementation, taking a table snapshot does not preserve the mob data. This issue will make sure that when a snapshot is taken, mob data is properly preserved and is restorable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11644) External MOB compaction tools
[ https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11644: --- Summary: External MOB compaction tools (was: External tools) External MOB compaction tools - Key: HBASE-11644 URL: https://issues.apache.org/jira/browse/HBASE-11644 Project: HBase Issue Type: Sub-task Components: Compaction, master Reporter: Jingcheng Du Assignee: Jingcheng Du The MOB files are involved in the HBase compaction. It means there's no chance to delete and merge the MOB files. The external tools do this, one is a cleaner to clean the MOB files that are expired (by TTL and minVersions), the other one is a sweep tool to clean the deleted Cells in HBase and merge small files into bigger ones. These tools are triggered by users. Besides, the cleaner could be a chore in HMaster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11645) Snapshot for MOB
[ https://issues.apache.org/jira/browse/HBASE-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11645: --- Description: Add snapshot support for MOB. In the initial implementation, taking a table snapshot does not preserve the mob data. This issue will make sure that when a snapshot is taken, mob data is properly preserved and is restorable. (was: Add snapshot support for MOB) Snapshot for MOB Key: HBASE-11645 URL: https://issues.apache.org/jira/browse/HBASE-11645 Project: HBase Issue Type: Sub-task Components: snapshots Reporter: Jingcheng Du Assignee: Jingcheng Du Add snapshot support for MOB. In the initial implementation, taking a table snapshot does not preserve the mob data. This issue will make sure that when a snapshot is taken, mob data is properly preserved and is restorable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11644) External MOB compaction tools
[ https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086324#comment-14086324 ] Jonathan Hsieh commented on HBASE-11644: I reorganized the description to make it easier to follow. External MOB compaction tools - Key: HBASE-11644 URL: https://issues.apache.org/jira/browse/HBASE-11644 Project: HBase Issue Type: Sub-task Components: Compaction, master Reporter: Jingcheng Du Assignee: Jingcheng Du The MOB files are involved in the HBase compaction. It means there's no chance to delete and merge the MOB files. The external tools do this, one is a cleaner to clean the MOB files that are expired (by TTL and minVersions), the other one is a sweep tool to clean the deleted Cells in HBase and merge small files into bigger ones. These tools are triggered by users. Besides, the cleaner could be a chore in HMaster. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11644) External MOB compaction tools
[ https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11644: --- Description: Form the design doc, mob files are not involved in the normal HBase compaction process. This means deleted mobs would still take up space and that we never really merge mob files that accrue over time. Currently, MOBs depend on two external tools: 1) A TTL cleaner that removes mobs that have passed their TTL or exceeded minVersions. 2) A 'sweep tool' cleaner that remove mobs that have had their references deleted and merges small files into larger ones. Today the tools are triggered by admins. The longer term goal would be to integrate them into hbase such that by default mobs are cleaned. The tools will be preserved however so that advanced admins can disable automatic cleanups and manually trigger these compaction like operaitons. #1 would likely be a chore in the master while #2 requires some design work to integrate into hbase. was:The MOB files are involved in the HBase compaction. It means there's no chance to delete and merge the MOB files. The external tools do this, one is a cleaner to clean the MOB files that are expired (by TTL and minVersions), the other one is a sweep tool to clean the deleted Cells in HBase and merge small files into bigger ones. These tools are triggered by users. Besides, the cleaner could be a chore in HMaster. External MOB compaction tools - Key: HBASE-11644 URL: https://issues.apache.org/jira/browse/HBASE-11644 Project: HBase Issue Type: Sub-task Components: Compaction, master Reporter: Jingcheng Du Assignee: Jingcheng Du Form the design doc, mob files are not involved in the normal HBase compaction process. This means deleted mobs would still take up space and that we never really merge mob files that accrue over time. Currently, MOBs depend on two external tools: 1) A TTL cleaner that removes mobs that have passed their TTL or exceeded minVersions. 2) A 'sweep tool' cleaner that remove mobs that have had their references deleted and merges small files into larger ones. Today the tools are triggered by admins. The longer term goal would be to integrate them into hbase such that by default mobs are cleaned. The tools will be preserved however so that advanced admins can disable automatic cleanups and manually trigger these compaction like operaitons. #1 would likely be a chore in the master while #2 requires some design work to integrate into hbase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11644) External MOB compaction tools
[ https://issues.apache.org/jira/browse/HBASE-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11644: --- Description: From the design doc, mob files are not involved in the normal HBase compaction process. This means deleted mobs would still take up space and that we never really merge mob files that accrue over time. Currently, MOBs depend on two external tools: 1) A TTL cleaner that removes mobs that have passed their TTL or exceeded minVersions. 2) A 'sweep tool' cleaner that remove mobs that have had their references deleted and merges small files into larger ones. Today the tools are triggered by admins. The longer term goal would be to integrate them into hbase such that by default mobs are cleaned. The tools will be preserved however so that advanced admins can disable automatic cleanups and manually trigger these compaction like operaitons. #1 would likely be a chore in the master while #2 requires some design work to integrate into hbase. was: Form the design doc, mob files are not involved in the normal HBase compaction process. This means deleted mobs would still take up space and that we never really merge mob files that accrue over time. Currently, MOBs depend on two external tools: 1) A TTL cleaner that removes mobs that have passed their TTL or exceeded minVersions. 2) A 'sweep tool' cleaner that remove mobs that have had their references deleted and merges small files into larger ones. Today the tools are triggered by admins. The longer term goal would be to integrate them into hbase such that by default mobs are cleaned. The tools will be preserved however so that advanced admins can disable automatic cleanups and manually trigger these compaction like operaitons. #1 would likely be a chore in the master while #2 requires some design work to integrate into hbase. External MOB compaction tools - Key: HBASE-11644 URL: https://issues.apache.org/jira/browse/HBASE-11644 Project: HBase Issue Type: Sub-task Components: Compaction, master Reporter: Jingcheng Du Assignee: Jingcheng Du From the design doc, mob files are not involved in the normal HBase compaction process. This means deleted mobs would still take up space and that we never really merge mob files that accrue over time. Currently, MOBs depend on two external tools: 1) A TTL cleaner that removes mobs that have passed their TTL or exceeded minVersions. 2) A 'sweep tool' cleaner that remove mobs that have had their references deleted and merges small files into larger ones. Today the tools are triggered by admins. The longer term goal would be to integrate them into hbase such that by default mobs are cleaned. The tools will be preserved however so that advanced admins can disable automatic cleanups and manually trigger these compaction like operaitons. #1 would likely be a chore in the master while #2 requires some design work to integrate into hbase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11646) Handle the MOB in compaction
[ https://issues.apache.org/jira/browse/HBASE-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086330#comment-14086330 ] Jonathan Hsieh commented on HBASE-11646: reorganized the description to make it easier to follow Handle the MOB in compaction Key: HBASE-11646 URL: https://issues.apache.org/jira/browse/HBASE-11646 Project: HBase Issue Type: Sub-task Components: Compaction Reporter: Jingcheng Du Assignee: Jingcheng Du For those MOB Cells loaded by the bulk load, they're saved in HBase. We need handle them in HBase compaction to write them to the MOB files. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11646) Handle the MOB in compaction
[ https://issues.apache.org/jira/browse/HBASE-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-11646: --- Description: In the updated MOB design however, admins can set CF level thresholds that would force cell values the threshold to use the MOB write path instead of the traditional path. There are two cases where mobs need to interact with this threshold 1) How do we handle the case when the threshold size is changed? 2) Today, you can bulkload hfiles that contain MOBs. These cells will work as normal inside hbase. Unfortunately the cells with MOBs in them will never benefit form the MOB write path. The proposal here is to modify compaction in mob enabled cf's such that the threshold value is honored with compactions. This handles case #1 -- elements that should be moved out of the normal hfiles get 'compacted' into refs and mob hfiles, and values that should be pulled into the cf get derefed and written out wholy in the compaction. For case #2, we can maintain the same behavior and compaction would move data into the mob writepath/lifecycle. was:For those MOB Cells loaded by the bulk load, they're saved in HBase. We need handle them in HBase compaction to write them to the MOB files. Handle the MOB in compaction Key: HBASE-11646 URL: https://issues.apache.org/jira/browse/HBASE-11646 Project: HBase Issue Type: Sub-task Components: Compaction Reporter: Jingcheng Du Assignee: Jingcheng Du In the updated MOB design however, admins can set CF level thresholds that would force cell values the threshold to use the MOB write path instead of the traditional path. There are two cases where mobs need to interact with this threshold 1) How do we handle the case when the threshold size is changed? 2) Today, you can bulkload hfiles that contain MOBs. These cells will work as normal inside hbase. Unfortunately the cells with MOBs in them will never benefit form the MOB write path. The proposal here is to modify compaction in mob enabled cf's such that the threshold value is honored with compactions. This handles case #1 -- elements that should be moved out of the normal hfiles get 'compacted' into refs and mob hfiles, and values that should be pulled into the cf get derefed and written out wholy in the compaction. For case #2, we can maintain the same behavior and compaction would move data into the mob writepath/lifecycle. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carter updated HBASE-11657:
---
Attachment: HBASE_11657.patch
Put HTable region methods in an interface
-
Key: HBASE-11657
URL: https://issues.apache.org/jira/browse/HBASE-11657
Project: HBase
Issue Type: Improvement
Affects Versions: 0.99.0
Reporter: Carter
Assignee: Carter
Fix For: 0.99.0
Attachments: HBASE_11657.patch
Most of the HTable methods are now abstracted by HTableInterface, with the
notable exception of the following methods that pertain to region metadata:
{code}
HRegionLocation getRegionLocation(final String row)
HRegionLocation getRegionLocation(final byte [] row)
HRegionLocation getRegionLocation(final byte [] row, boolean reload)
byte [][] getStartKeys()
byte[][] getEndKeys()
Pairbyte[][],byte[][] getStartEndKeys()
void clearRegionCache()
{code}
and a default scope method which maybe should be bundled with the others:
{code}
ListRegionLocations listRegionLocations()
{code}
Since the consensus seems to be that these would muddy HTableInterface with
non-core functionality, where should it go? MapReduce looks up the region
boundaries, so it needs to be exposed somewhere.
Let me throw out a straw man to start the conversation. I propose:
{code}
org.apache.hadoop.hbase.client.HRegionInterface
{code}
Have HTable implement this interface. Also add these methods to HConnection:
{code}
HRegionInterface getTableRegion(TableName tableName)
HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
{code}
[~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carter updated HBASE-11657:
---
Status: Patch Available (was: Open)
RegionLocator it is.
Put HTable region methods in an interface
-
Key: HBASE-11657
URL: https://issues.apache.org/jira/browse/HBASE-11657
Project: HBase
Issue Type: Improvement
Affects Versions: 0.99.0
Reporter: Carter
Assignee: Carter
Fix For: 0.99.0
Attachments: HBASE_11657.patch
Most of the HTable methods are now abstracted by HTableInterface, with the
notable exception of the following methods that pertain to region metadata:
{code}
HRegionLocation getRegionLocation(final String row)
HRegionLocation getRegionLocation(final byte [] row)
HRegionLocation getRegionLocation(final byte [] row, boolean reload)
byte [][] getStartKeys()
byte[][] getEndKeys()
Pairbyte[][],byte[][] getStartEndKeys()
void clearRegionCache()
{code}
and a default scope method which maybe should be bundled with the others:
{code}
ListRegionLocations listRegionLocations()
{code}
Since the consensus seems to be that these would muddy HTableInterface with
non-core functionality, where should it go? MapReduce looks up the region
boundaries, so it needs to be exposed somewhere.
Let me throw out a straw man to start the conversation. I propose:
{code}
org.apache.hadoop.hbase.client.HRegionInterface
{code}
Have HTable implement this interface. Also add these methods to HConnection:
{code}
HRegionInterface getTableRegion(TableName tableName)
HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
{code}
[~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11666) Enforce JDK7 javac for builds on branch-1 and master
[ https://issues.apache.org/jira/browse/HBASE-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-11666: --- Resolution: Fixed Fix Version/s: 2.0.0 0.99.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Integrated to branch-1 and trunk. Thanks for the patch, Sean. Enforce JDK7 javac for builds on branch-1 and master Key: HBASE-11666 URL: https://issues.apache.org/jira/browse/HBASE-11666 Project: HBase Issue Type: Task Components: build, documentation Affects Versions: 1.0.0, 2.0.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Fix For: 0.99.0, 2.0.0 Attachments: HBASE_11666-v1.patch, HBASE_11666-v2.patch, HBASE_11666-v3.patch Per [the discussion on JDK versions|http://search-hadoop.com/m/DHED4Zlz0R1], we require Java 7 for 1.0+ (also we have code that only compiles under Java 7). Right now, if you attempt to build with JDK6, Maven will happily oblige and then give you a cannot find symbol error about a Java 7 class we use. We should * update the Build HBase guide to reference our supported Java versions docs * update our pom so that Maven will correctly complain about jdk versions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086370#comment-14086370
]
Andrew Purtell commented on HBASE-11667:
bq. The client cannot track what the server is doing unless the server tells it
what it did (i.e. how far it got with its scan). I don't think we can recover
if there is no way to know which state to recover to. All the client can know
without help from the server (this particular scan) if last startkey of the
last region. I tried to only use that information, but it turns out that does
not work. Interesting problem :)
The problem with your patch was the client ended up handing duplicate rows to
the application layer because we removed a kludge. The client is in the
position of observing the Results it is passing up. I wonder if it is possible
to detect the 'skipFirst' condition and handle it without relying on the
current server response, since we see at least in this case a coprocessor can
cause the server to lie. (Of course, we can always declare that to be an
invalid thing to do.) I am not looking at code when saying this, just thinking
out loud.
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11642) EOL 0.96
[ https://issues.apache.org/jira/browse/HBASE-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086386#comment-14086386 ] Lars Hofhansl commented on HBASE-11642: --- Changed the text to this for now: We suggest downloading the current stable release. The 0.98.x series is the current stable release, it supercedes 0.94.x and 0.96.x. [~stack] feel free to change again. :) EOL 0.96 Key: HBASE-11642 URL: https://issues.apache.org/jira/browse/HBASE-11642 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Do the work to EOL 0.96. + No more patches on 0.96 + Remove 0.96 from downloads. + If user has issue with 0.96 and needs fix, fix it in 0.98 and have the user upgrade to get the fix. + Write email to user list stating 0.96 has been EOL'd September 1st? And add notice to refguide. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11589) AccessControlException handling in HBase rpc server and client. AccessControlException should be a not retriable exception
[
https://issues.apache.org/jira/browse/HBASE-11589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086390#comment-14086390
]
Andrew Purtell commented on HBASE-11589:
We already have an AccessDeniedException. This patch adds a new
AccessControlException. The naming is too close, we should have one or the
other. Given that client code, if security is active, is already expecting ADE,
I think that's the best choice.
AccessControlException handling in HBase rpc server and client.
AccessControlException should be a not retriable exception
--
Key: HBASE-11589
URL: https://issues.apache.org/jira/browse/HBASE-11589
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.98.3
Environment: SLES 11 SP1
Reporter: Kashif J S
Assignee: Qiang Tian
Attachments: hbase-11589-master-v1.patch, hbase-11589-master.patch
RPC server does not handle the AccessControlException thrown by
authorizeConnection failure properly and in return sends IOException to the
HBase client.
Ultimately the client does retries and gets RetriesExhaustedException but
does not getting any link or information or stack trace about
AccessControlException.
In short summary, upon inspection of RPCServer.java, it seems
for the Listener, the Reader read code as below does not handle
AccessControlException
{noformat}
void doRead(….
…..
…..
try {
count = c.readAndProcess(); // This readAndProcess method throws
AccessControlException from processOneRpc(byte[] buf) which is not handled ?
} catch (InterruptedException ieo) {
throw ieo;
} catch (Exception e) {
LOG.warn(getName() + : count of bytes read: + count, e);
count = -1; //so that the (count 0) block is executed
}
{noformat}
Below is the client logs if authorizeConnection throws AccessControlException:
2014-07-24 19:40:58,768 INFO [main]
client.HConnectionManager$HConnectionImplementation: getMaster attempt 7 of 7
failed; no more retrying.
com.google.protobuf.ServiceException: java.io.IOException: Call to
host-10-18-40-101/10.18.40.101:6 failed on local exception:
java.io.EOFException
at
org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674)
at
org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885)
at
org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3302)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3329)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:605)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:496)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:430)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:311)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:59)
at
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at
[jira] [Resolved] (HBASE-11675) Major compactions change query results
[ https://issues.apache.org/jira/browse/HBASE-11675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-11675. Resolution: Invalid That section of the manual doesn't mention a *bug*, it documents some semantics that clients need to consider. Major compactions change query results -- Key: HBASE-11675 URL: https://issues.apache.org/jira/browse/HBASE-11675 Project: HBase Issue Type: Bug Reporter: chendihao The bug mentioned in http://hbase.apache.org/book.html 5.9.2.2. Major compactions change query results “...create three cell versions at t1, t2 and t3, with a maximum-versions setting of 2. So when getting all versions, only the values at t2 and t3 will be returned. But if you delete the version at t2 or t3, the one at t1 will appear again. Obviously, once a major compaction has run, such behavior will not be the case anymore...” -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086408#comment-14086408
]
Lars Hofhansl commented on HBASE-11667:
---
It seems we either have to delay up to an entire region worth of data from
delivering up the caller, or we have to know up to what point we have delivered
and that means keeping track of the last key we have delivered (it does not
need to a real key, but it needs to be unique). Maybe I am being trapped in how
things are right now...?
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11642) EOL 0.96
[ https://issues.apache.org/jira/browse/HBASE-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086381#comment-14086381 ] Lars Hofhansl commented on HBASE-11642: --- Yes. It should be a 100% that all new users should go to 0.98 (and soon 1.0, yeah!) Completely with you there. The stable pointer now points to 0.98. Yeah, we need to fix the text... I'll do that now. EOL 0.96 Key: HBASE-11642 URL: https://issues.apache.org/jira/browse/HBASE-11642 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Do the work to EOL 0.96. + No more patches on 0.96 + Remove 0.96 from downloads. + If user has issue with 0.96 and needs fix, fix it in 0.98 and have the user upgrade to get the fix. + Write email to user list stating 0.96 has been EOL'd September 1st? And add notice to refguide. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11666) Enforce JDK7 javac for builds on branch-1 and master
[ https://issues.apache.org/jira/browse/HBASE-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086359#comment-14086359 ] Hudson commented on HBASE-11666: FAILURE: Integrated in HBase-TRUNK #5370 (See [https://builds.apache.org/job/HBase-TRUNK/5370/]) HBASE-11666 Enforce JDK7 javac for builds on branch-1 and master (Sean Busbey) (tedyu: rev b158900b682888f88f74537c43032be36e567b04) * bin/hbase-config.sh * pom.xml * conf/hbase-env.sh * conf/hbase-env.cmd * src/main/docbkx/developer.xml Enforce JDK7 javac for builds on branch-1 and master Key: HBASE-11666 URL: https://issues.apache.org/jira/browse/HBASE-11666 Project: HBase Issue Type: Task Components: build, documentation Affects Versions: 1.0.0, 2.0.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Fix For: 0.99.0, 2.0.0 Attachments: HBASE_11666-v1.patch, HBASE_11666-v2.patch, HBASE_11666-v3.patch Per [the discussion on JDK versions|http://search-hadoop.com/m/DHED4Zlz0R1], we require Java 7 for 1.0+ (also we have code that only compiles under Java 7). Right now, if you attempt to build with JDK6, Maven will happily oblige and then give you a cannot find symbol error about a Java 7 class we use. We should * update the Build HBase guide to reference our supported Java versions docs * update our pom so that Maven will correctly complain about jdk versions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086383#comment-14086383
]
Hadoop QA commented on HBASE-11657:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12659895/HBASE_11657.patch
against trunk revision .
ATTACHMENT ID: 12659895
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.ipc.TestIPC
org.apache.hadoop.hbase.master.TestClockSkewDetection
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10299//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10299//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10299//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10299//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10299//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10299//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10299//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10299//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10299//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10299//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10299//console
This message is automatically generated.
Put HTable region methods in an interface
-
Key: HBASE-11657
URL: https://issues.apache.org/jira/browse/HBASE-11657
Project: HBase
Issue Type: Improvement
Affects Versions: 0.99.0
Reporter: Carter
Assignee: Carter
Fix For: 0.99.0
Attachments: HBASE_11657.patch
Most of the HTable methods are now abstracted by HTableInterface, with the
notable exception of the following methods that pertain to region metadata:
{code}
HRegionLocation getRegionLocation(final String row)
HRegionLocation getRegionLocation(final byte [] row)
HRegionLocation getRegionLocation(final byte [] row, boolean reload)
byte [][] getStartKeys()
byte[][] getEndKeys()
Pairbyte[][],byte[][] getStartEndKeys()
void clearRegionCache()
{code}
and a default scope method which maybe should be bundled with the others:
{code}
ListRegionLocations listRegionLocations()
{code}
Since the consensus seems to be that these would muddy HTableInterface with
non-core functionality, where should it go? MapReduce looks up the region
boundaries, so it needs to be exposed somewhere.
Let me throw out a straw man to start the conversation. I propose:
{code}
org.apache.hadoop.hbase.client.HRegionInterface
{code}
Have HTable implement this interface. Also add these methods to HConnection:
{code}
HRegionInterface getTableRegion(TableName tableName)
HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
{code}
[~stack], [~ndimiduk], [~enis], thoughts?
[jira] [Commented] (HBASE-11666) Enforce JDK7 javac for builds on branch-1 and master
[
https://issues.apache.org/jira/browse/HBASE-11666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086411#comment-14086411
]
Hadoop QA commented on HBASE-11666:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12659892/HBASE_11666-v3.patch
against trunk revision .
ATTACHMENT ID: 12659892
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
org.apache.hadoop.hbase.regionserver.TestRegionReplicas
org.apache.hadoop.hbase.client.TestReplicasClient
org.apache.hadoop.hbase.master.TestRestartCluster
org.apache.hadoop.hbase.regionserver.TestHRegionBusyWait
org.apache.hadoop.hbase.TestIOFencing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10298//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10298//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10298//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10298//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10298//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10298//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10298//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10298//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10298//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10298//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10298//console
This message is automatically generated.
Enforce JDK7 javac for builds on branch-1 and master
Key: HBASE-11666
URL: https://issues.apache.org/jira/browse/HBASE-11666
Project: HBase
Issue Type: Task
Components: build, documentation
Affects Versions: 1.0.0, 2.0.0
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Minor
Fix For: 0.99.0, 2.0.0
Attachments: HBASE_11666-v1.patch, HBASE_11666-v2.patch,
HBASE_11666-v3.patch
Per [the discussion on JDK versions|http://search-hadoop.com/m/DHED4Zlz0R1],
we require Java 7 for 1.0+ (also we have code that only compiles under Java
7).
Right now, if you attempt to build with JDK6, Maven will happily oblige and
then give you a cannot find symbol error about a Java 7 class we use.
We should
* update the Build HBase guide to reference our supported Java versions docs
* update our pom so that Maven will correctly complain about jdk versions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086414#comment-14086414
]
Lars Hofhansl commented on HBASE-11667:
---
So commit the java doc change for now?
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11667) Comment ClientScanner logic for NSREs.
[
https://issues.apache.org/jira/browse/HBASE-11667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086416#comment-14086416
]
Andrew Purtell commented on HBASE-11667:
bq. or we have to know up to what point we have delivered and that means
keeping track of the last key we have delivered
I would say this.
Comment ClientScanner logic for NSREs.
--
Key: HBASE-11667
URL: https://issues.apache.org/jira/browse/HBASE-11667
Project: HBase
Issue Type: Improvement
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.99.0, 2.0.0, 0.94.23, 0.98.6
Attachments: 11667-0.94.txt, 11667-doc-0.94.txt, 11667-trunk.txt,
HBASE-11667-0.98.patch, IntegrationTestBigLinkedListWithRegionMovement.patch
We ran into an issue with Phoenix where a RegionObserver coprocessor
intercepts a scan and returns an aggregate (in this case a count) with a fake
row key. It turns out this does not work when the {{ClientScanner}}
encounters NSREs, as it uses the last key it saw to reset the scanner to try
again (which in this case would be the fake key).
While this is arguably a rare case and one could also argue that a region
observer just shouldn't do this... While looking at {{ClientScanner}}'s code
I found this logic not necessary.
A NSRE occurred because we contacted a region server with a key that it no
longer hosts. This is the start key, so it is always correct to retry with
this same key. That simplifies the ClientScanner logic and also make this
sort of coprocessors possible,
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11674) LoadIncrementalHFiles should be more verbose after unrecoverable error
[ https://issues.apache.org/jira/browse/HBASE-11674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086419#comment-14086419 ] Andrew Purtell commented on HBASE-11674: lgtm LoadIncrementalHFiles should be more verbose after unrecoverable error -- Key: HBASE-11674 URL: https://issues.apache.org/jira/browse/HBASE-11674 Project: HBase Issue Type: Improvement Components: mapreduce Affects Versions: 0.98.5 Reporter: Jan Lukavsky Assignee: Jan Lukavsky Attachments: HBASE-11674-ii.patch, HBASE-11674.patch LoadIncrementalHFiles should give more information after failure to load data to regionserver. Currently, it logs only Encountered unrecoverable error from region server, but doesn't give information about * which region server it talked to * which was the region that failed to load data In order to help understand what is going on, the log should contain both these information. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11657) Put HTable region methods in an interface
[
https://issues.apache.org/jira/browse/HBASE-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086430#comment-14086430
]
Carter commented on HBASE-11657:
No deltas between tests I ran on master and the patch.
No new test since this is simply an interface. I'll refactor other tests to
use the interface in the future.
I have a general question for the group. Why is _getRegionLocation_ okay, but
this one is deprecated?
{code}
@Deprecated
public ListHRegionLocation getRegionsInRange(final byte [] startKey, final
byte [] endKey, final boolean reload)
{code}
Does this seem like an appropriate method now in RegionLocator? If this
remains deprecated, what's the correct alternative for clients that use it?
Put HTable region methods in an interface
-
Key: HBASE-11657
URL: https://issues.apache.org/jira/browse/HBASE-11657
Project: HBase
Issue Type: Improvement
Affects Versions: 0.99.0
Reporter: Carter
Assignee: Carter
Fix For: 0.99.0
Attachments: HBASE_11657.patch
Most of the HTable methods are now abstracted by HTableInterface, with the
notable exception of the following methods that pertain to region metadata:
{code}
HRegionLocation getRegionLocation(final String row)
HRegionLocation getRegionLocation(final byte [] row)
HRegionLocation getRegionLocation(final byte [] row, boolean reload)
byte [][] getStartKeys()
byte[][] getEndKeys()
Pairbyte[][],byte[][] getStartEndKeys()
void clearRegionCache()
{code}
and a default scope method which maybe should be bundled with the others:
{code}
ListRegionLocations listRegionLocations()
{code}
Since the consensus seems to be that these would muddy HTableInterface with
non-core functionality, where should it go? MapReduce looks up the region
boundaries, so it needs to be exposed somewhere.
Let me throw out a straw man to start the conversation. I propose:
{code}
org.apache.hadoop.hbase.client.HRegionInterface
{code}
Have HTable implement this interface. Also add these methods to HConnection:
{code}
HRegionInterface getTableRegion(TableName tableName)
HRegionInterface getTableRegion(TableName tableName, ExecutorService pool)
{code}
[~stack], [~ndimiduk], [~enis], thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11671) TestEndToEndSplitTransaction fails on master
[
https://issues.apache.org/jira/browse/HBASE-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086437#comment-14086437
]
Jimmy Xiang commented on HBASE-11671:
-
I prefer the first patch. What do you think?
TestEndToEndSplitTransaction fails on master
Key: HBASE-11671
URL: https://issues.apache.org/jira/browse/HBASE-11671
Project: HBase
Issue Type: Bug
Components: Region Assignment, test
Affects Versions: 0.99.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 0.99.0
Attachments: HBASE-11671-v2.patch, HBASE-11671.patch
#testMasterOpsWhileSplitting() fails, seems after enabling ZK-less assignment.
Here're the only log snippets which seem relevant.
{quote}
java.io.IOException: Failed to report opened region to master:
TestSplit,lll,1407218018712.030c51654a5cbe9e0489a50762bb9075.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1731)
at
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.testMasterOpsWhileSplitting(TestEndToEndSplitTransaction.java:130)
{quote}
{quote}
java.io.IOException: Cannot append; log is closed
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1146)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.appendNoSync(FSHLog.java:1120)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:266)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1211)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1158)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1514)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:478)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11606) Enable ZK-less region assignment by default
[ https://issues.apache.org/jira/browse/HBASE-11606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086438#comment-14086438 ] Jimmy Xiang commented on HBASE-11606: - Will fix those tests in a separate issue. Enable ZK-less region assignment by default --- Key: HBASE-11606 URL: https://issues.apache.org/jira/browse/HBASE-11606 Project: HBase Issue Type: Task Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 2.0.0 Attachments: hbase-11606.patch Let's enable ZK-less region assignment by default in the master branch. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11606) Enable ZK-less region assignment by default
[
https://issues.apache.org/jira/browse/HBASE-11606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086453#comment-14086453
]
Jimmy Xiang commented on HBASE-11606:
-
I only see these test failures
{quote}
org.apache.hadoop.hbase.master.TestClockSkewDetection
org.apache.hadoop.hbase.procedure.TestProcedureManager
org.apache.hadoop.hbase.ipc.TestIPC
{quote}
They are ok locally, seems not related to this change.
Enable ZK-less region assignment by default
---
Key: HBASE-11606
URL: https://issues.apache.org/jira/browse/HBASE-11606
Project: HBase
Issue Type: Task
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 2.0.0
Attachments: hbase-11606.patch
Let's enable ZK-less region assignment by default in the master branch.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (HBASE-11671) TestEndToEndSplitTransaction fails on master
[
https://issues.apache.org/jira/browse/HBASE-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086437#comment-14086437
]
Jimmy Xiang edited comment on HBASE-11671 at 8/5/14 4:19 PM:
-
+1 on the second patch. Thanks.
was (Author: jxiang):
I prefer the first patch. What do you think?
TestEndToEndSplitTransaction fails on master
Key: HBASE-11671
URL: https://issues.apache.org/jira/browse/HBASE-11671
Project: HBase
Issue Type: Bug
Components: Region Assignment, test
Affects Versions: 0.99.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 0.99.0
Attachments: HBASE-11671-v2.patch, HBASE-11671.patch
#testMasterOpsWhileSplitting() fails, seems after enabling ZK-less assignment.
Here're the only log snippets which seem relevant.
{quote}
java.io.IOException: Failed to report opened region to master:
TestSplit,lll,1407218018712.030c51654a5cbe9e0489a50762bb9075.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1731)
at
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.testMasterOpsWhileSplitting(TestEndToEndSplitTransaction.java:130)
{quote}
{quote}
java.io.IOException: Cannot append; log is closed
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1146)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.appendNoSync(FSHLog.java:1120)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:266)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1211)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1158)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1514)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:478)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11671) TestEndToEndSplitTransaction fails on master
[
https://issues.apache.org/jira/browse/HBASE-11671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-11671:
Resolution: Fixed
Fix Version/s: (was: 0.99.0)
2.0.0
1.0.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Integrated v2 into master and branch 1. Thanks a lot for the patch.
TestEndToEndSplitTransaction fails on master
Key: HBASE-11671
URL: https://issues.apache.org/jira/browse/HBASE-11671
Project: HBase
Issue Type: Bug
Components: Region Assignment, test
Affects Versions: 0.99.0
Reporter: Mikhail Antonov
Assignee: Mikhail Antonov
Fix For: 1.0.0, 2.0.0
Attachments: HBASE-11671-v2.patch, HBASE-11671.patch
#testMasterOpsWhileSplitting() fails, seems after enabling ZK-less assignment.
Here're the only log snippets which seem relevant.
{quote}
java.io.IOException: Failed to report opened region to master:
TestSplit,lll,1407218018712.030c51654a5cbe9e0489a50762bb9075.
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1731)
at
org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction.testMasterOpsWhileSplitting(TestEndToEndSplitTransaction.java:130)
{quote}
{quote}
java.io.IOException: Cannot append; log is closed
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1146)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.appendNoSync(FSHLog.java:1120)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:266)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1211)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1158)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1514)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:478)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{quote}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11606) Enable ZK-less region assignment by default
[ https://issues.apache.org/jira/browse/HBASE-11606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086488#comment-14086488 ] Devaraj Das commented on HBASE-11606: - The full suite never ran. Look at https://builds.apache.org/job/HBase-TRUNK/5352/testReport/ .. you will see 1411 tests ran and took 4 minutes or so to run (top right hand corner). Under normal circumstances 3000+ tests should run. Enable ZK-less region assignment by default --- Key: HBASE-11606 URL: https://issues.apache.org/jira/browse/HBASE-11606 Project: HBase Issue Type: Task Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 2.0.0 Attachments: hbase-11606.patch Let's enable ZK-less region assignment by default in the master branch. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11641) TestDistributedLogSplitting.testMasterStartsUpWithLogSplittingWork fails frequently
[ https://issues.apache.org/jira/browse/HBASE-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-11641: -- Fix Version/s: 0.94.23 TestDistributedLogSplitting.testMasterStartsUpWithLogSplittingWork fails frequently --- Key: HBASE-11641 URL: https://issues.apache.org/jira/browse/HBASE-11641 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.94.23 Attachments: 11641-logs.txt, 11641.txt 3 out of last four run of my RC jenkins build failed in this test. Might be related to HBASE-11041 too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11642) EOL 0.96
[ https://issues.apache.org/jira/browse/HBASE-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086499#comment-14086499 ] stack commented on HBASE-11642: --- [~lhofhansl] lgtm EOL 0.96 Key: HBASE-11642 URL: https://issues.apache.org/jira/browse/HBASE-11642 Project: HBase Issue Type: Task Reporter: stack Assignee: stack Do the work to EOL 0.96. + No more patches on 0.96 + Remove 0.96 from downloads. + If user has issue with 0.96 and needs fix, fix it in 0.98 and have the user upgrade to get the fix. + Write email to user list stating 0.96 has been EOL'd September 1st? And add notice to refguide. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11641) TestDistributedLogSplitting.testMasterStartsUpWithLogSplittingWork fails frequently
[ https://issues.apache.org/jira/browse/HBASE-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086502#comment-14086502 ] Lars Hofhansl commented on HBASE-11641: --- K... Lemme commit that. 0.94 only (this is already fixed in 0.98 and later) TestDistributedLogSplitting.testMasterStartsUpWithLogSplittingWork fails frequently --- Key: HBASE-11641 URL: https://issues.apache.org/jira/browse/HBASE-11641 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.94.23 Attachments: 11641-logs.txt, 11641.txt 3 out of last four run of my RC jenkins build failed in this test. Might be related to HBASE-11041 too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11670) Build PDF of Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086506#comment-14086506 ] stack commented on HBASE-11670: --- You tried it [~misty] ? It don't come out the best. Long lines run off the edge, etc. Build PDF of Ref Guide -- Key: HBASE-11670 URL: https://issues.apache.org/jira/browse/HBASE-11670 Project: HBase Issue Type: Task Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Priority: Minor -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11208) Remove the hbase.hstore.blockingStoreFiles setting
[ https://issues.apache.org/jira/browse/HBASE-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086514#comment-14086514 ] Sergey Shelukhin commented on HBASE-11208: -- Makes sense to me... Remove the hbase.hstore.blockingStoreFiles setting -- Key: HBASE-11208 URL: https://issues.apache.org/jira/browse/HBASE-11208 Project: HBase Issue Type: Brainstorming Components: Compaction, regionserver Affects Versions: 0.99.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0 It's a little bit of a provocation, but the rational is: - there are some bugs around the delayed flush. For example, if the periodic scheduler has asked for a delayed flush, and that we need to flush, we will have to wait - if the number of WAL files increases, we won't flush immediately if the blockingFile number has been reached. This impacts the MTTR. - We don't write to limit the compaction impact, but they are many cases where we would want to flush anyway, as the writes cannot wait. - this obviously leads to huge write latency peaks. So I'm questioning this setting, it leads to multiple intricate cases, unpredictable write latency, and looks like a workaround for compaction performances. With all the work done on compaction, I think we can get rid of it. A solution in the middle would be to deprecate it and to set it to a large value... Any opinion before I shoot :-) ? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11641) TestDistributedLogSplitting.testMasterStartsUpWithLogSplittingWork fails frequently
[ https://issues.apache.org/jira/browse/HBASE-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-11641. --- Resolution: Fixed Assignee: Lars Hofhansl Committed. Crossing fingers. TestDistributedLogSplitting.testMasterStartsUpWithLogSplittingWork fails frequently --- Key: HBASE-11641 URL: https://issues.apache.org/jira/browse/HBASE-11641 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.23 Attachments: 11641-logs.txt, 11641.txt 3 out of last four run of my RC jenkins build failed in this test. Might be related to HBASE-11041 too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11208) Remove the hbase.hstore.blockingStoreFiles setting
[ https://issues.apache.org/jira/browse/HBASE-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086523#comment-14086523 ] Lars Hofhansl commented on HBASE-11208: --- There are two issues: OOMs and read performance. We'd then need another way than to configure HBase for read vs. write loads. This setting is what puts an upper bound on acceptable read performance by limiting the number of HFiles a scan has to look through - at the expense of blocking writers if they write faster than the IO subsystem can absorb. The key is some mechanism to force HBase to slow clients down or block them completely from writing if some guaranteed read performance is desired. Remove the hbase.hstore.blockingStoreFiles setting -- Key: HBASE-11208 URL: https://issues.apache.org/jira/browse/HBASE-11208 Project: HBase Issue Type: Brainstorming Components: Compaction, regionserver Affects Versions: 0.99.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0 It's a little bit of a provocation, but the rational is: - there are some bugs around the delayed flush. For example, if the periodic scheduler has asked for a delayed flush, and that we need to flush, we will have to wait - if the number of WAL files increases, we won't flush immediately if the blockingFile number has been reached. This impacts the MTTR. - We don't write to limit the compaction impact, but they are many cases where we would want to flush anyway, as the writes cannot wait. - this obviously leads to huge write latency peaks. So I'm questioning this setting, it leads to multiple intricate cases, unpredictable write latency, and looks like a workaround for compaction performances. With all the work done on compaction, I think we can get rid of it. A solution in the middle would be to deprecate it and to set it to a large value... Any opinion before I shoot :-) ? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11678) BucketCache ramCache fills heap after running a few hours
stack created HBASE-11678: - Summary: BucketCache ramCache fills heap after running a few hours Key: HBASE-11678 URL: https://issues.apache.org/jira/browse/HBASE-11678 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Testing BucketCache, my heap filled after running for hours. Dumping heap, culprit is the ramCache Map in BucketCache. Tried running with more writer threads but made no difference. Trying to figure now how our accounting is going wonky. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11447) Proposal for a generic transaction API for HBase
[ https://issues.apache.org/jira/browse/HBASE-11447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086540#comment-14086540 ] Gary Helmling commented on HBASE-11447: --- [~jamestaylor] [~john-deroo] Sorry, I was away for the week with no connectivity when James commented. I'm putting together some comments based on the current proposal and the approach that we've taken with Tephra (https://github.com/continuuity/tephra). Proposal for a generic transaction API for HBase Key: HBASE-11447 URL: https://issues.apache.org/jira/browse/HBASE-11447 Project: HBase Issue Type: New Feature Components: Client Affects Versions: 1.0.0 Environment: Any. Reporter: John de Roo Priority: Minor Labels: features, newbie Fix For: 1.0.0 Attachments: Proposal for a common transactional API for HBase v0.3_1.pdf, Proposal for a common transactional API for HBase v0.4_1.pdf, Proposal for a common transactional API for HBase v0.5.pdf, Re Proposal for a generic transaction API for HBase.htm HBase transaction management today is provided by a number of products, each implementing a different API, each having different strengths. The lack of a common API for transactional interfaces means that applications need to be coded to work with a specific Transaction Manager. This proposal outlines an API which, if implemented by the different Transaction Manager vendors would provide stability and choice to HBase application developers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11678) BucketCache ramCache fills heap after running a few hours
[ https://issues.apache.org/jira/browse/HBASE-11678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11678: -- Component/s: BlockCache Affects Version/s: 2.0.0 0.99.0 0.98.5 BucketCache ramCache fills heap after running a few hours - Key: HBASE-11678 URL: https://issues.apache.org/jira/browse/HBASE-11678 Project: HBase Issue Type: Bug Components: BlockCache Affects Versions: 0.99.0, 0.98.5, 2.0.0 Reporter: stack Assignee: stack Testing BucketCache, my heap filled after running for hours. Dumping heap, culprit is the ramCache Map in BucketCache. Tried running with more writer threads but made no difference. Trying to figure now how our accounting is going wonky. -- This message was sent by Atlassian JIRA (v6.2#6252)
