[jira] [Commented] (HBASE-8316) JoinedHeap for essential column families should reseek instead of seek
[
https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627528#comment-13627528
]
Hadoop QA commented on HBASE-8316:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577958/8316-0.96.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5239//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5239//console
This message is automatically generated.
JoinedHeap for essential column families should reseek instead of seek
--
Key: HBASE-8316
URL: https://issues.apache.org/jira/browse/HBASE-8316
Project: HBase
Issue Type: Sub-task
Components: Filters, Performance, regionserver
Reporter: Lars Hofhansl
Fix For: 0.98.0, 0.94.7, 0.95.1
Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt
This was raised by the Phoenix team. During a profiling session we noticed
that catching the joinedHeap up to the current rows via seek causes a
performance regression, which makes the joinedHeap only efficient when either
a high or low percentage is matched by the filter.
(High is fine, because the joinedHeap will not get behind as often and does
not need to be caught up, low is fine, because the seek isn't happening
frequently).
In our tests we found that the solution is quite simple: Replace seek with
reseek. Patch coming soon.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8252) Regions by Region Server table in Master's table view needs styling
[ https://issues.apache.org/jira/browse/HBASE-8252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627534#comment-13627534 ] Hudson commented on HBASE-8252: --- Integrated in HBase-TRUNK #4048 (See [https://builds.apache.org/job/HBase-TRUNK/4048/]) HBASE-8252 Regions by Region Server table in Master's table view needs styling (Revision 1466233) Result = FAILURE eclark : Files : * /hbase/trunk/hbase-server/src/main/resources/hbase-webapps/master/table.jsp Regions by Region Server table in Master's table view needs styling --- Key: HBASE-8252 URL: https://issues.apache.org/jira/browse/HBASE-8252 Project: HBase Issue Type: Bug Components: UI Affects Versions: 0.95.0 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Trivial Labels: noob Fix For: 0.98.0, 0.95.1 Attachments: HBASE-8252-0.patch, HBASE-8252-1.patch, Table t.png Need to add class=table to the table for region counts per region server on table display page. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6330) TestImportExport has been failing against hadoop 0.23/2.0 profile
[ https://issues.apache.org/jira/browse/HBASE-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627535#comment-13627535 ] Hudson commented on HBASE-6330: --- Integrated in HBase-TRUNK #4048 (See [https://builds.apache.org/job/HBase-TRUNK/4048/]) HBASE-6330 TestImportExport has been failing against hadoop 0.23/2.0 profile (Revision 1466256) Result = FAILURE jmhsieh : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/Export.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java TestImportExport has been failing against hadoop 0.23/2.0 profile - Key: HBASE-6330 URL: https://issues.apache.org/jira/browse/HBASE-6330 Project: HBase Issue Type: Sub-task Components: test Affects Versions: 0.94.1, 0.95.2 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Labels: hadoop-2.0 Fix For: 0.98.0, 0.95.1 Attachments: hbase-6330-94.patch, hbase-6330-trunk.patch, hbase-6330-v2.patch, hbase-6330.v4.patch See HBASE-5876. I'm going to commit the v3 patches under this name since there has been two months (my bad) since the first half was committed and found to be incomplte. --- 4/9/13 Updated - this will take the patch from HBASE-8258 to fix this specific problem. The umbrella that used to be HBASE-8258 is now handled with HBASE-6891. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7791) Compaction USER_PRIORITY is slightly broken
[
https://issues.apache.org/jira/browse/HBASE-7791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627536#comment-13627536
]
Hudson commented on HBASE-7791:
---
Integrated in HBase-TRUNK #4048 (See
[https://builds.apache.org/job/HBase-TRUNK/4048/])
HBASE-7791 Compaction USER_PRIORITY is slightly broken (Revision 1466267)
Result = FAILURE
sershe :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFileManager.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
Compaction USER_PRIORITY is slightly broken
---
Key: HBASE-7791
URL: https://issues.apache.org/jira/browse/HBASE-7791
Project: HBase
Issue Type: Bug
Components: Compaction
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Priority: Minor
Attachments: HBASE-7791-v0.patch, HBASE-7791-v1.patch,
HBASE-7791-v2.patch
The code to get compaction priority is as such:
{code} public int getCompactPriority(int priority) {
// If this is a user-requested compaction, leave this at the highest
priority
if(priority == Store.PRIORITY_USER) {
return Store.PRIORITY_USER;
} else {
return this.blockingStoreFileCount - this.storefiles.size();
}
}
{code}.
PRIORITY_USER is 1.
The priorities are compared as numbers in HRegion, so compactions of blocking
stores will override user priority (probably intended); also, if you have
blockingFiles minus one, your priority is suddenly PRIORITY_USER, which may
cause at least this:
LOG.debug(Warning, compacting more than +
comConf.getMaxFilesToCompact() +
files because of a user-requested major compaction);
as well as some misleading logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8316) JoinedHeap for essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627539#comment-13627539 ] Lars Hofhansl commented on HBASE-8316: -- Looking good. Will await the perf numbers from Phoenix ([~giacomotaylor]), and then commit if all looks right. (Don't think the tests have to be redone with requestSeek specifically, as it will only be better than reseek) JoinedHeap for essential column families should reseek instead of seek -- Key: HBASE-8316 URL: https://issues.apache.org/jira/browse/HBASE-8316 Project: HBase Issue Type: Sub-task Components: Filters, Performance, regionserver Reporter: Lars Hofhansl Fix For: 0.98.0, 0.94.7, 0.95.1 Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt This was raised by the Phoenix team. During a profiling session we noticed that catching the joinedHeap up to the current rows via seek causes a performance regression, which makes the joinedHeap only efficient when either a high or low percentage is matched by the filter. (High is fine, because the joinedHeap will not get behind as often and does not need to be caught up, low is fine, because the seek isn't happening frequently). In our tests we found that the solution is quite simple: Replace seek with reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8153) TestJoinedScanners fails occasionally
[
https://issues.apache.org/jira/browse/HBASE-8153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627544#comment-13627544
]
Nicolas Liochon commented on HBASE-8153:
Still there, for example on trunk build #4008
{noformat}
Failed 16 actions: FailedServerException: 16 times,
Stacktrace
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 16
actions: FailedServerException: 16 times,
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$Process$BatchErrors.makeException(HConnectionManager.java:2238)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$Process$BatchErrors.rethrowIfAny(HConnectionManager.java:)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$Process.processBatchCallback(HConnectionManager.java:2205)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$Process.access$1000(HConnectionManager.java:1973)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1962)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1941)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1035)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:694)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:682)
at
org.apache.hadoop.hbase.regionserver.TestJoinedScanners.testJoinedScanners(TestJoinedScanners.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
Logs are there:
https://builds.apache.org/job/HBase-TRUNK/ws/trunk/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.TestJoinedScanners-output.txt
[~xieliang007] are you working on this? If you spent some time on this, do you
have any idea on the root cause?
TestJoinedScanners fails occasionally
-
Key: HBASE-8153
URL: https://issues.apache.org/jira/browse/HBASE-8153
Project: HBase
Issue Type: Test
Components: test
Affects Versions: 0.98.0
Reporter: Liang Xie
see
https://builds.apache.org/job/PreCommit-HBASE-Build/4908//testReport/org.apache.hadoop.hbase.regionserver/TestJoinedScanners/testJoinedScanners/
and HBASE-7845
It should be a false alame, this's a placeholder to figure out the root cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8316) JoinedHeap for essential column families should reseek instead of seek
[
https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627547#comment-13627547
]
Hadoop QA commented on HBASE-8316:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577962/8316-trunk.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5240//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5240//console
This message is automatically generated.
JoinedHeap for essential column families should reseek instead of seek
--
Key: HBASE-8316
URL: https://issues.apache.org/jira/browse/HBASE-8316
Project: HBase
Issue Type: Sub-task
Components: Filters, Performance, regionserver
Reporter: Lars Hofhansl
Fix For: 0.98.0, 0.94.7, 0.95.1
Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt
This was raised by the Phoenix team. During a profiling session we noticed
that catching the joinedHeap up to the current rows via seek causes a
performance regression, which makes the joinedHeap only efficient when either
a high or low percentage is matched by the filter.
(High is fine, because the joinedHeap will not get behind as often and does
not need to be caught up, low is fine, because the seek isn't happening
frequently).
In our tests we found that the solution is quite simple: Replace seek with
reseek. Patch coming soon.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4811) Support reverse Scan
[
https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627551#comment-13627551
]
Liang Xie commented on HBASE-4811:
--
Previous HBase-4811-0.94.3modified.txt didn't have a backward-compatibility, we
can bump a SCAN version, like HBASE-6250's style:
e.g.:
{code}
--- main/java/org/apache/hadoop/hbase/client/Scan.java (revision 88835)
+++ main/java/org/apache/hadoop/hbase/client/Scan.java (working copy)
@@ -85,6 +85,7 @@
private static final String ISOLATION_LEVEL = _isolationlevel_;
private static final byte SCAN_VERSION = (byte)2;
+ private static final byte SCAN_REVERSED_VERSION = (byte)3;
private byte [] startRow = HConstants.EMPTY_START_ROW;
private byte [] stopRow = HConstants.EMPTY_END_ROW;
private int maxVersions = 1;
@@ -577,7 +578,7 @@
public void readFields(final DataInput in)
throws IOException {
int version = in.readByte();
-if (version (int)SCAN_VERSION) {
+if (version SCAN_REVERSED_VERSION) {
throw new IOException(version not supported);
}
this.startRow = Bytes.readByteArray(in);
@@ -586,7 +587,9 @@
this.batch = in.readInt();
this.caching = in.readInt();
this.cacheBlocks = in.readBoolean();
-this.reverse = in.readBoolean();
+if (version = SCAN_REVERSED_VERSION) {
+ this.reverse = in.readBoolean();
+}
if(in.readBoolean()) {
this.filter =
(Filter)createForName(Bytes.toString(Bytes.readByteArray(in)));
this.filter.readFields(in);
@@ -614,14 +617,20 @@
public void write(final DataOutput out)
throws IOException {
-out.writeByte(SCAN_VERSION);
+if (reverse) {
+ out.writeByte(SCAN_REVERSED_VERSION);
+} else {
+ out.writeByte(SCAN_VERSION);
+}
Bytes.writeByteArray(out, this.startRow);
Bytes.writeByteArray(out, this.stopRow);
out.writeInt(this.maxVersions);
out.writeInt(this.batch);
out.writeInt(this.caching);
out.writeBoolean(this.cacheBlocks);
-out.writeBoolean(this.reverse);
+if (reverse) {
+ out.writeBoolean(this.reverse);
+}
if(this.filter == null) {
out.writeBoolean(false);
} else {
{code}
Support reverse Scan
Key: HBASE-4811
URL: https://issues.apache.org/jira/browse/HBASE-4811
Project: HBase
Issue Type: Improvement
Components: Client
Affects Versions: 0.20.6
Reporter: John Carrino
Attachments: HBase-4811-0.94.3modified.txt
All the documentation I find about HBase says that if you want forward and
reverse scans you should just build 2 tables and one be ascending and one
descending. Is there a fundamental reason that HBase only supports forward
Scan? It seems like a lot of extra space overhead and coding overhead (to
keep them in sync) to support 2 tables.
I am assuming this has been discussed before, but I can't find the
discussions anywhere about it or why it would be infeasible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6680) Allow configurable/multiple split log threads per RS.
[ https://issues.apache.org/jira/browse/HBASE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627575#comment-13627575 ] binlijin commented on HBASE-6680: - Should we backport this issue? Allow configurable/multiple split log threads per RS. - Key: HBASE-6680 URL: https://issues.apache.org/jira/browse/HBASE-6680 Project: HBase Issue Type: Improvement Reporter: Amitanand Aiyer Assignee: Amitanand Aiyer Priority: Minor Attachments: HBASE-6680-94.patch For rack failure case, it seems like there are multiple batches of split logs to be processed. Allow a configurable number of split log worker per RS, so that we split more logs quickly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6680) Allow configurable/multiple split log threads per RS.
[ https://issues.apache.org/jira/browse/HBASE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HBASE-6680: Attachment: HBASE-6680-94.patch Allow configurable/multiple split log threads per RS. - Key: HBASE-6680 URL: https://issues.apache.org/jira/browse/HBASE-6680 Project: HBase Issue Type: Improvement Reporter: Amitanand Aiyer Assignee: Amitanand Aiyer Priority: Minor Attachments: HBASE-6680-94.patch For rack failure case, it seems like there are multiple batches of split logs to be processed. Allow a configurable number of split log worker per RS, so that we split more logs quickly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8312) TestCompactionState - still timeouting
[ https://issues.apache.org/jira/browse/HBASE-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-8312: --- Resolution: Fixed Fix Version/s: 0.95.1 0.98.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) TestCompactionState - still timeouting -- Key: HBASE-8312 URL: https://issues.apache.org/jira/browse/HBASE-8312 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.95.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.95.1 Attachments: 8312.v1.patch Some work was done in HBASE-7560, it seems it's not enough. Let's increase these timeouts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-8316: -- Summary: JoinedHeap for non essential column families should reseek instead of seek (was: JoinedHeap for essential column families should reseek instead of seek) JoinedHeap for non essential column families should reseek instead of seek -- Key: HBASE-8316 URL: https://issues.apache.org/jira/browse/HBASE-8316 Project: HBase Issue Type: Sub-task Components: Filters, Performance, regionserver Reporter: Lars Hofhansl Fix For: 0.98.0, 0.94.7, 0.95.1 Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt This was raised by the Phoenix team. During a profiling session we noticed that catching the joinedHeap up to the current rows via seek causes a performance regression, which makes the joinedHeap only efficient when either a high or low percentage is matched by the filter. (High is fine, because the joinedHeap will not get behind as often and does not need to be caught up, low is fine, because the seek isn't happening frequently). In our tests we found that the solution is quite simple: Replace seek with reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627622#comment-13627622 ] Anoop Sam John commented on HBASE-8316: --- Ya. requestSeek() in this case will help with HFile's max timestamp based lazy seeks. There won't be any bloom usage here in this case. +1 JoinedHeap for non essential column families should reseek instead of seek -- Key: HBASE-8316 URL: https://issues.apache.org/jira/browse/HBASE-8316 Project: HBase Issue Type: Sub-task Components: Filters, Performance, regionserver Reporter: Lars Hofhansl Fix For: 0.98.0, 0.94.7, 0.95.1 Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt This was raised by the Phoenix team. During a profiling session we noticed that catching the joinedHeap up to the current rows via seek causes a performance regression, which makes the joinedHeap only efficient when either a high or low percentage is matched by the filter. (High is fine, because the joinedHeap will not get behind as often and does not need to be caught up, low is fine, because the seek isn't happening frequently). In our tests we found that the solution is quite simple: Replace seek with reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4811) Support reverse Scan
[ https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627625#comment-13627625 ] ramkrishna.s.vasudevan commented on HBASE-4811: --- I will check this patch today or tomorrow. Looks interesting. Support reverse Scan Key: HBASE-4811 URL: https://issues.apache.org/jira/browse/HBASE-4811 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.20.6 Reporter: John Carrino Attachments: HBase-4811-0.94.3modified.txt All the documentation I find about HBase says that if you want forward and reverse scans you should just build 2 tables and one be ascending and one descending. Is there a fundamental reason that HBase only supports forward Scan? It seems like a lot of extra space overhead and coding overhead (to keep them in sync) to support 2 tables. I am assuming this has been discussed before, but I can't find the discussions anywhere about it or why it would be infeasible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8317) Seek returns wrong result with PREFIX_TREE Encoding
chunhui shen created HBASE-8317: --- Summary: Seek returns wrong result with PREFIX_TREE Encoding Key: HBASE-8317 URL: https://issues.apache.org/jira/browse/HBASE-8317 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: chunhui shen Assignee: chunhui shen TestPrefixTreeEncoding#testSeekWithFixedData from the patch could reproduce the bug. An example of the bug case: Suppose the following rows: 1.row3/c1:q1/ 2.row3/c1:q2/ 3.row3/c1:q3/ 4.row4/c1:q1/ 5.row4/c1:q2/ After seeking the row 'row30', the expected peek KV is row4/c1:q1/, but actual is row3/c1:q1/. I just fix this bug case in the patch, Maybe we can do more for other potential problems if anyone is familiar with the code of PREFIX_TREE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8317) Seek returns wrong result with PREFIX_TREE Encoding
[ https://issues.apache.org/jira/browse/HBASE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-8317: Attachment: hbase-trunk-8317.patch Seek returns wrong result with PREFIX_TREE Encoding --- Key: HBASE-8317 URL: https://issues.apache.org/jira/browse/HBASE-8317 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: chunhui shen Assignee: chunhui shen Attachments: hbase-trunk-8317.patch TestPrefixTreeEncoding#testSeekWithFixedData from the patch could reproduce the bug. An example of the bug case: Suppose the following rows: 1.row3/c1:q1/ 2.row3/c1:q2/ 3.row3/c1:q3/ 4.row4/c1:q1/ 5.row4/c1:q2/ After seeking the row 'row30', the expected peek KV is row4/c1:q1/, but actual is row3/c1:q1/. I just fix this bug case in the patch, Maybe we can do more for other potential problems if anyone is familiar with the code of PREFIX_TREE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8313) Add Bloom filter testing for HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-8313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-8313: --- Resolution: Fixed Fix Version/s: 0.95.1 0.94.7 Status: Resolved (was: Patch Available) Add Bloom filter testing for HFileOutputFormat -- Key: HBASE-8313 URL: https://issues.apache.org/jira/browse/HBASE-8313 Project: HBase Issue Type: Bug Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.94.7, 0.95.1 Attachments: HBASE-8313-94.patch, HBASE-8313-v0.patch HBASE-3776 added Bloom Filter Support to the HFileOutputFormat, but there's no test to verify that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7247) Assignment performances decreased by 50% because of regionserver.OpenRegionHandler#tickleOpening
[ https://issues.apache.org/jira/browse/HBASE-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627662#comment-13627662 ] Nicolas Liochon commented on HBASE-7247: bq. This sync happens every time? Ain't it really expensive? More expensive that what used to be there (a write?). Why we need it? We already had the sync previously. We're just saving the final write. The sync can be expensive, I don't know if it's always expensive or sometimes optimized by ZK. My test hides its cost, because it's a single ZK node and the master is on the same node. A real life system with 5 ZK would be more realistic. In any case, saving the write is good. bq. Does this clear any existing watch? No, it does not set a watcher but should not impact existing ones. bq. Does the content of retransitionNodeOpening deserve to be generalized and moved back into transitionNode but with a flag for whether to write the new state or not? I don't know. Do we have a lot of case like this one? Globally, the performance problem is that we check the state before writing, hence we sync and we read. The next optimization I'm thinking about is to know that we were the previous writer, or that we read the previous version already, so we can send the write without syncing/reading, relying only on ZooKeeper versions. But it's easy to add bugs when doing this. Assignment performances decreased by 50% because of regionserver.OpenRegionHandler#tickleOpening Key: HBASE-7247 URL: https://issues.apache.org/jira/browse/HBASE-7247 Project: HBase Issue Type: Improvement Components: master, Region Assignment, regionserver Affects Versions: 0.95.2 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.95.1 Attachments: 7247.v1.patch, 7247.v2.patch The regionserver.OpenRegionHandler#tickleOpening updates the region znode as Do this so master doesn't timeout this region-in-transition.. However, on the usual test, this makes the assignment time of 1500 regions goes from 70s to 100s, that is, we're 50% slower because of this. More generally, ZooKeper commits to disk all the data update, and this takes time. Using it to provide a keep alive seems overkill. At the very list, it could be made asynchronous. I'm not sure how necessary these updates are required (I need to go deeper in the internal, feedback welcome), but it seems very important to optimize this... The trival fix would be to make this optional. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8317) Seek returns wrong result with PREFIX_TREE Encoding
[
https://issues.apache.org/jira/browse/HBASE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627684#comment-13627684
]
ramkrishna.s.vasudevan commented on HBASE-8317:
---
@Chunhui
I am not aware of this Prefix. But one thing i noticed is that,
for batch0_row2* it is behaving bit different for others bit different.
Even after this patch if you see the matching entries it goes like this,
{code}
batch0_row0/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row0/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row1/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row1/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row3/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row3/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row4/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row4/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row5/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row5/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row6/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row6/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row7/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row7/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row8/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row8/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row9/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row9/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row2/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row20/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
#batch0_row20/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
batch0_row21/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
#batch0_row21/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
batch0_row22/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
#batch0_row22/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
batch0_row23/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
#batch0_row23/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
batch0_row24/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
#batch0_row24/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
batch0_row25/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
#batch0_row25/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
batch0_row26/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
#batch0_row26/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
batch0_row27/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
#batch0_row27/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
batch0_row28/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
#batch0_row28/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
batch0_row29/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
#batch0_row29/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=27/mvcc=0
batch0_row4/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row4/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
batch0_row4/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
#batch0_row4/EncodingTestCF:col0/LATEST_TIMESTAMP/Put/vlen=26/mvcc=0
[jira] [Commented] (HBASE-8282) User triggered flushes does not allow compaction to get triggered even if compaction criteria is met
[
https://issues.apache.org/jira/browse/HBASE-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627687#comment-13627687
]
ramkrishna.s.vasudevan commented on HBASE-8282:
---
@Sergey
What do you mean here? So you say that we need to do builder.setFlushed() if
something was flushed rather than setting the return value from flushCache()?
Because the return value infact says about compaction and not flushing, am i
right?
User triggered flushes does not allow compaction to get triggered even if
compaction criteria is met
Key: HBASE-8282
URL: https://issues.apache.org/jira/browse/HBASE-8282
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.98.0, 0.94.7, 0.95.1
Attachments: HBASE-8282_trunk.patch
Observe the below logs
{code}
2013-04-04 17:43:45,825 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Flushing
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9.
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., current region
memstore size 42.3 M
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., commencing wait for
mvcc, flushsize=44388936
2013-04-04 17:43:45,826 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2013-04-04 17:43:45,831 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating
file=hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
with permission=rwxrwxrwx
2013-04-04 17:43:45,834 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
Initialized with CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2013-04-04 17:43:45,835 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
Delete Family Bloom filter type for
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e:
CompoundBloomFilterWriter
2013-04-04 17:43:46,074 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
NO General Bloom and NO DeleteFamily was added to HFile
(hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e)
2013-04-04 17:43:46,074 INFO org.apache.hadoop.hbase.regionserver.HStore:
Flushed , sequenceid=1051817, memsize=42.3 M, into tmp file
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
2013-04-04 17:43:46,093 DEBUG
org.apache.hadoop.hbase.regionserver.HRegionFileSystem: Committing store file
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/.tmp/7020db24b38246a5818e7eb4e130ad9e
as
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/f1/7020db24b38246a5818e7eb4e130ad9e
2013-04-04 17:43:46,103 INFO org.apache.hadoop.hbase.regionserver.HStore:
Added
hdfs://localhost:9010/hbase/MyTable/f6792086ad3518dee244e7bf2761a1f9/f1/7020db24b38246a5818e7eb4e130ad9e,
entries=54911, sequenceid=1051817, filesize=35.1 M
2013-04-04 17:43:46,103 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~42.3 M/44388936, currentsize=0/0 for region
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9. in 278ms,
sequenceid=-1, compaction requested=true
2013-04-04 17:43:56,335 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
Stats: total=394.09 MB, free=3.61 GB, max=4.00 GB, blocks=5263,
accesses=244882, hits=42988, hitRatio=17.55%, , cachingAccesses=49869,
cachingHits=24251, cachingHitsRatio=48.63%, , evictions=0, evicted=20349,
evictedPerRun=Infinity
2013-04-04 17:44:31,119 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Flushing
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9.
2013-04-04 17:44:31,120 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., current region
memstore size 42.7 M
2013-04-04 17:44:31,120 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting
MyTable,,1365111467842.f6792086ad3518dee244e7bf2761a1f9., commencing wait for
mvcc, flushsize=44764248
2013-04-04 17:44:31,120 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2013-04-04 17:44:31,136 DEBUG org.apache.hadoop.hbase.util.FSUtils: Creating
[jira] [Assigned] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reassigned HBASE-8316: - Assignee: Ted Yu JoinedHeap for non essential column families should reseek instead of seek -- Key: HBASE-8316 URL: https://issues.apache.org/jira/browse/HBASE-8316 Project: HBase Issue Type: Sub-task Components: Filters, Performance, regionserver Reporter: Lars Hofhansl Assignee: Ted Yu Fix For: 0.98.0, 0.94.7, 0.95.1 Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt This was raised by the Phoenix team. During a profiling session we noticed that catching the joinedHeap up to the current rows via seek causes a performance regression, which makes the joinedHeap only efficient when either a high or low percentage is matched by the filter. (High is fine, because the joinedHeap will not get behind as often and does not need to be caught up, low is fine, because the seek isn't happening frequently). In our tests we found that the solution is quite simple: Replace seek with reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8318) TableOutputFormat.TableRecordWriter should accept Increments
Jean-Marc Spaggiari created HBASE-8318: -- Summary: TableOutputFormat.TableRecordWriter should accept Increments Key: HBASE-8318 URL: https://issues.apache.org/jira/browse/HBASE-8318 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari TableOutputFormat.TableRecordWriter can take Puts and Deletes but it should also accept Increments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8318) TableOutputFormat.TableRecordWriter should accept Increments
[ https://issues.apache.org/jira/browse/HBASE-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-8318: --- Status: Patch Available (was: Open) Small patch attached. TableOutputFormat.TableRecordWriter should accept Increments Key: HBASE-8318 URL: https://issues.apache.org/jira/browse/HBASE-8318 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Attachments: HBASE-8318-v0-trunk.patch TableOutputFormat.TableRecordWriter can take Puts and Deletes but it should also accept Increments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8318) TableOutputFormat.TableRecordWriter should accept Increments
[ https://issues.apache.org/jira/browse/HBASE-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-8318: --- Attachment: HBASE-8318-v0-trunk.patch TableOutputFormat.TableRecordWriter should accept Increments Key: HBASE-8318 URL: https://issues.apache.org/jira/browse/HBASE-8318 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Attachments: HBASE-8318-v0-trunk.patch TableOutputFormat.TableRecordWriter can take Puts and Deletes but it should also accept Increments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8313) Add Bloom filter testing for HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-8313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627773#comment-13627773 ] Hudson commented on HBASE-8313: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #491 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/491/]) HBASE-8313 Add Bloom filter testing for HFileOutputFormat (Revision 1466412) Result = FAILURE mbertozzi : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java Add Bloom filter testing for HFileOutputFormat -- Key: HBASE-8313 URL: https://issues.apache.org/jira/browse/HBASE-8313 Project: HBase Issue Type: Bug Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.94.7, 0.95.1 Attachments: HBASE-8313-94.patch, HBASE-8313-v0.patch HBASE-3776 added Bloom Filter Support to the HFileOutputFormat, but there's no test to verify that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8312) TestCompactionState - still timeouting
[ https://issues.apache.org/jira/browse/HBASE-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627774#comment-13627774 ] Hudson commented on HBASE-8312: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #491 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/491/]) HBASE-8312 TestCompactionState - still timeouting (Revision 1466345) Result = FAILURE nkeywal : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionState.java TestCompactionState - still timeouting -- Key: HBASE-8312 URL: https://issues.apache.org/jira/browse/HBASE-8312 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.95.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.95.1 Attachments: 8312.v1.patch Some work was done in HBASE-7560, it seems it's not enough. Let's increase these timeouts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-1936: --- Attachment: trunk-1936_v2.patch ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Jimmy Xiang Labels: noob Attachments: cp_from_hdfs.patch, HBASE-1936-trunk(forReview).patch, trunk-1936.patch, trunk-1936_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-1936: --- Status: Open (was: Patch Available) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Jimmy Xiang Labels: noob Attachments: cp_from_hdfs.patch, HBASE-1936-trunk(forReview).patch, trunk-1936.patch, trunk-1936_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-1936: --- Fix Version/s: 0.95.1 0.98.0 Status: Patch Available (was: Open) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Jimmy Xiang Labels: noob Fix For: 0.98.0, 0.95.1 Attachments: cp_from_hdfs.patch, HBASE-1936-trunk(forReview).patch, trunk-1936.patch, trunk-1936_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8313) Add Bloom filter testing for HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-8313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627848#comment-13627848 ] Hudson commented on HBASE-8313: --- Integrated in HBase-TRUNK #4049 (See [https://builds.apache.org/job/HBase-TRUNK/4049/]) HBASE-8313 Add Bloom filter testing for HFileOutputFormat (Revision 1466412) Result = SUCCESS mbertozzi : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java Add Bloom filter testing for HFileOutputFormat -- Key: HBASE-8313 URL: https://issues.apache.org/jira/browse/HBASE-8313 Project: HBase Issue Type: Bug Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.94.7, 0.95.1 Attachments: HBASE-8313-94.patch, HBASE-8313-v0.patch HBASE-3776 added Bloom Filter Support to the HFileOutputFormat, but there's no test to verify that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8312) TestCompactionState - still timeouting
[ https://issues.apache.org/jira/browse/HBASE-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627849#comment-13627849 ] Hudson commented on HBASE-8312: --- Integrated in HBase-TRUNK #4049 (See [https://builds.apache.org/job/HBase-TRUNK/4049/]) HBASE-8312 TestCompactionState - still timeouting (Revision 1466345) Result = SUCCESS nkeywal : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionState.java TestCompactionState - still timeouting -- Key: HBASE-8312 URL: https://issues.apache.org/jira/browse/HBASE-8312 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.95.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.95.1 Attachments: 8312.v1.patch Some work was done in HBASE-7560, it seems it's not enough. Let's increase these timeouts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8314) HLogSplitter can retry to open a 0-length hlog file
[
https://issues.apache.org/jira/browse/HBASE-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627857#comment-13627857
]
Jimmy Xiang commented on HBASE-8314:
Sure, part of the log is attached. I have observed the same issue in several 92
releases.
HLogSplitter can retry to open a 0-length hlog file
---
Key: HBASE-8314
URL: https://issues.apache.org/jira/browse/HBASE-8314
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: region-server.log
In case a HLog file is of size 0, and it is under recovery, HLogSplitter will
fail to open it since it can get the file length, therefore, master can't
start.
{noformat}
java.io.IOException: Cannot obtain block length for LocatedBlock{...;
getBlockSize()=0; corrupt=false; offset=0; locs=[...]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:238)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:182)
at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:124)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:117)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1080)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8314) HLogSplitter can retry to open a 0-length hlog file
[
https://issues.apache.org/jira/browse/HBASE-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-8314:
---
Attachment: region-server.log
HLogSplitter can retry to open a 0-length hlog file
---
Key: HBASE-8314
URL: https://issues.apache.org/jira/browse/HBASE-8314
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: region-server.log
In case a HLog file is of size 0, and it is under recovery, HLogSplitter will
fail to open it since it can get the file length, therefore, master can't
start.
{noformat}
java.io.IOException: Cannot obtain block length for LocatedBlock{...;
getBlockSize()=0; corrupt=false; offset=0; locs=[...]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:238)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:182)
at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:124)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:117)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1080)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8165: - Attachment: 8165v5.txt Fix javadoc warnings (Were in generated files -- 2.5 seems to treat the comments in .protos differently) Update our protobuf to 2.5 from 2.4.1 - Key: HBASE-8165 URL: https://issues.apache.org/jira/browse/HBASE-8165 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.95.1 Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8312) TestCompactionState - still timeouting
[ https://issues.apache.org/jira/browse/HBASE-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627894#comment-13627894 ] Hudson commented on HBASE-8312: --- Integrated in hbase-0.95 #138 (See [https://builds.apache.org/job/hbase-0.95/138/]) HBASE-8312 TestCompactionState - still timeouting (Revision 1466344) Result = SUCCESS nkeywal : Files : * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionState.java TestCompactionState - still timeouting -- Key: HBASE-8312 URL: https://issues.apache.org/jira/browse/HBASE-8312 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.95.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.95.1 Attachments: 8312.v1.patch Some work was done in HBASE-7560, it seems it's not enough. Let's increase these timeouts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8312) TestCompactionState - still timeouting
[ https://issues.apache.org/jira/browse/HBASE-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627938#comment-13627938 ] Hudson commented on HBASE-8312: --- Integrated in hbase-0.95-on-hadoop2 #64 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/64/]) HBASE-8312 TestCompactionState - still timeouting (Revision 1466344) Result = FAILURE nkeywal : Files : * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionState.java TestCompactionState - still timeouting -- Key: HBASE-8312 URL: https://issues.apache.org/jira/browse/HBASE-8312 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.95.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.95.1 Attachments: 8312.v1.patch Some work was done in HBASE-7560, it seems it's not enough. Let's increase these timeouts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8314) HLogSplitter can retry to open a 0-length hlog file
[
https://issues.apache.org/jira/browse/HBASE-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627946#comment-13627946
]
ramkrishna.s.vasudevan commented on HBASE-8314:
---
Retry should work here..May be latest HDFS versions handle it better.
HLogSplitter can retry to open a 0-length hlog file
---
Key: HBASE-8314
URL: https://issues.apache.org/jira/browse/HBASE-8314
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: region-server.log
In case a HLog file is of size 0, and it is under recovery, HLogSplitter will
fail to open it since it can get the file length, therefore, master can't
start.
{noformat}
java.io.IOException: Cannot obtain block length for LocatedBlock{...;
getBlockSize()=0; corrupt=false; offset=0; locs=[...]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:238)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:182)
at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:124)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:117)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1080)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7658) grant with an empty string as permission should throw an exception
[
https://issues.apache.org/jira/browse/HBASE-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-7658:
---
Attachment: HBASE-7658-v1.patch
grant with an empty string as permission should throw an exception
--
Key: HBASE-7658
URL: https://issues.apache.org/jira/browse/HBASE-7658
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.94.4, 0.95.2
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Attachments: HBASE-7658-v0.patch, HBASE-7658-v1.patch
If someone specify an empty permission
{code}grant 'user', ''{code}
AccessControlLists.addUserPermission() output a log message and doesn't
change the permission, but the user doesn't know about it.
{code}
if ((actions == null) || (actions.length == 0)) {
LOG.warn(No actions associated with user
'+Bytes.toString(userPerm.getUser())+');
return;
}
{code}
I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7658) grant with an empty string as permission should throw an exception
[
https://issues.apache.org/jira/browse/HBASE-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-7658:
---
Affects Version/s: (was: 0.94.4)
Status: Patch Available (was: Open)
grant with an empty string as permission should throw an exception
--
Key: HBASE-7658
URL: https://issues.apache.org/jira/browse/HBASE-7658
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.2
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Attachments: HBASE-7658-v0.patch, HBASE-7658-v1.patch
If someone specify an empty permission
{code}grant 'user', ''{code}
AccessControlLists.addUserPermission() output a log message and doesn't
change the permission, but the user doesn't know about it.
{code}
if ((actions == null) || (actions.length == 0)) {
LOG.warn(No actions associated with user
'+Bytes.toString(userPerm.getUser())+');
return;
}
{code}
I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[
https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627953#comment-13627953
]
Hadoop QA commented on HBASE-8165:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578020/8165v5.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 24 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5243//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5243//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5243//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5243//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5243//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5243//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5243//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5243//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5243//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5243//console
This message is automatically generated.
Update our protobuf to 2.5 from 2.4.1
-
Key: HBASE-8165
URL: https://issues.apache.org/jira/browse/HBASE-8165
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Fix For: 0.95.1
Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt
Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses
making a builder.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8319) hbase-it tests are run when you ask to run all tests -- it should be that you have to ask explicitly to run them
stack created HBASE-8319: Summary: hbase-it tests are run when you ask to run all tests -- it should be that you have to ask explicitly to run them Key: HBASE-8319 URL: https://issues.apache.org/jira/browse/HBASE-8319 Project: HBase Issue Type: Task Reporter: stack Priority: Critical Fix For: 0.95.1 Up on trunk and on 0.95 apache builds, Sergey noticed that hbase-it tests are running. Up to this, the convention was that you had to explicitly ask that they run but that changed somehow recently. These tests are heavyweight, take a long time to complete, and are very likely to fail up on the apache infra (which is what we want of them but not as part of the general proofing build). For now the tests have been artificially disabled up on builds.apache.org but their inclusion likely frustrates joe blow trying to do a local hbase packaging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7658) grant with an empty string as permission should throw an exception
[
https://issues.apache.org/jira/browse/HBASE-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627961#comment-13627961
]
Andrew Purtell commented on HBASE-7658:
---
+1 on v1 patch
grant with an empty string as permission should throw an exception
--
Key: HBASE-7658
URL: https://issues.apache.org/jira/browse/HBASE-7658
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.2
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Attachments: HBASE-7658-v0.patch, HBASE-7658-v1.patch
If someone specify an empty permission
{code}grant 'user', ''{code}
AccessControlLists.addUserPermission() output a log message and doesn't
change the permission, but the user doesn't know about it.
{code}
if ((actions == null) || (actions.length == 0)) {
LOG.warn(No actions associated with user
'+Bytes.toString(userPerm.getUser())+');
return;
}
{code}
I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8165: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to 0.95 and trunk. Update our protobuf to 2.5 from 2.4.1 - Key: HBASE-8165 URL: https://issues.apache.org/jira/browse/HBASE-8165 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.95.1 Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627974#comment-13627974 ] Lars Hofhansl commented on HBASE-8316: -- Seems like this should be assigned to me. JoinedHeap for non essential column families should reseek instead of seek -- Key: HBASE-8316 URL: https://issues.apache.org/jira/browse/HBASE-8316 Project: HBase Issue Type: Sub-task Components: Filters, Performance, regionserver Reporter: Lars Hofhansl Assignee: Ted Yu Fix For: 0.98.0, 0.94.7, 0.95.1 Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt This was raised by the Phoenix team. During a profiling session we noticed that catching the joinedHeap up to the current rows via seek causes a performance regression, which makes the joinedHeap only efficient when either a high or low percentage is matched by the filter. (High is fine, because the joinedHeap will not get behind as often and does not need to be caught up, low is fine, because the seek isn't happening frequently). In our tests we found that the solution is quite simple: Replace seek with reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8266) Master cannot start if TableNotFoundException is thrown while partial table recovery
[
https://issues.apache.org/jira/browse/HBASE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627979#comment-13627979
]
ramkrishna.s.vasudevan commented on HBASE-8266:
---
Yes, but we need to clear the znode. In HBASE-5583 i have tried handling all
these scenarios but currently it was suggested that we can wait for HBASE-5487.
Master cannot start if TableNotFoundException is thrown while partial table
recovery
Key: HBASE-8266
URL: https://issues.apache.org/jira/browse/HBASE-8266
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.6, 0.95.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.98.0, 0.94.7, 0.95.1
Attachments: HBASE-8266_0.94.patch, HBASE-8266_1.patch,
HBASE-8266.patch
I was trying to create a table. The table creation failed
{code}
java.io.IOException: java.util.concurrent.ExecutionException:
java.lang.IllegalStateException: Could not instantiate a region instance.
at
org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
at
org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateHdfsRegions(CreateTableHandler.java:256)
at
org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:204)
at
org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:153)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:130)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.IllegalStateException: Could not instantiate a region instance.
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
... 7 more
Caused by: java.lang.IllegalStateException: Could not instantiate a region
instance.
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3765)
at
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:3870)
at
org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:106)
at
org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:103)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3762)
... 11 more
Caused by: java.lang.NoClassDefFoundError:
org/apache/hadoop/hbase/CompoundConfiguration$1
at
org.apache.hadoop.hbase.CompoundConfiguration.add(CompoundConfiguration.java:82)
at org.apache.hadoop.hbase.regionserver.HRegion.init(HRegion.java:438)
at org.apache.hadoop.hbase.regionserver.HRegion.init(HRegion.java:401)
... 16 more
{code}
Am not sure of the above failure. The same setup is able to create new
tables.
Now the table is already in ENABLING state. The master was restarted.
Now as the table was found in ENABLING state but not added to META the
EnableTableHandler
{code}
2013-04-03 18:33:03,850 FATAL org.apache.hadoop.hbase.master.HMaster:
Unhandled exception. Starting shutdown.
org.apache.hadoop.hbase.exceptions.TableNotFoundException: TestTable
at
org.apache.hadoop.hbase.master.handler.EnableTableHandler.prepare(EnableTableHandler.java:89)
at
org.apache.hadoop.hbase.master.AssignmentManager.recoverTableInEnablingState(AssignmentManager.java:2586)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:390)
at
[jira] [Commented] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627980#comment-13627980 ] Ted Yu commented on HBASE-8316: --- I am fine either way. JoinedHeap for non essential column families should reseek instead of seek -- Key: HBASE-8316 URL: https://issues.apache.org/jira/browse/HBASE-8316 Project: HBase Issue Type: Sub-task Components: Filters, Performance, regionserver Reporter: Lars Hofhansl Assignee: Ted Yu Fix For: 0.98.0, 0.94.7, 0.95.1 Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt This was raised by the Phoenix team. During a profiling session we noticed that catching the joinedHeap up to the current rows via seek causes a performance regression, which makes the joinedHeap only efficient when either a high or low percentage is matched by the filter. (High is fine, because the joinedHeap will not get behind as often and does not need to be caught up, low is fine, because the seek isn't happening frequently). In our tests we found that the solution is quite simple: Replace seek with reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8266) Master cannot start if TableNotFoundException is thrown while partial table recovery
[
https://issues.apache.org/jira/browse/HBASE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-8266:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to 0.94, 0.95 and trunk.
Thanks Lars, Bijieshan, Ted and Matteo for the review.
Master cannot start if TableNotFoundException is thrown while partial table
recovery
Key: HBASE-8266
URL: https://issues.apache.org/jira/browse/HBASE-8266
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.6, 0.95.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.98.0, 0.94.7, 0.95.1
Attachments: HBASE-8266_0.94.patch, HBASE-8266_1.patch,
HBASE-8266.patch
I was trying to create a table. The table creation failed
{code}
java.io.IOException: java.util.concurrent.ExecutionException:
java.lang.IllegalStateException: Could not instantiate a region instance.
at
org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:133)
at
org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateHdfsRegions(CreateTableHandler.java:256)
at
org.apache.hadoop.hbase.master.handler.CreateTableHandler.handleCreateTable(CreateTableHandler.java:204)
at
org.apache.hadoop.hbase.master.handler.CreateTableHandler.process(CreateTableHandler.java:153)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:130)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.util.concurrent.ExecutionException:
java.lang.IllegalStateException: Could not instantiate a region instance.
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:126)
... 7 more
Caused by: java.lang.IllegalStateException: Could not instantiate a region
instance.
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3765)
at
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:3870)
at
org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:106)
at
org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:103)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3762)
... 11 more
Caused by: java.lang.NoClassDefFoundError:
org/apache/hadoop/hbase/CompoundConfiguration$1
at
org.apache.hadoop.hbase.CompoundConfiguration.add(CompoundConfiguration.java:82)
at org.apache.hadoop.hbase.regionserver.HRegion.init(HRegion.java:438)
at org.apache.hadoop.hbase.regionserver.HRegion.init(HRegion.java:401)
... 16 more
{code}
Am not sure of the above failure. The same setup is able to create new
tables.
Now the table is already in ENABLING state. The master was restarted.
Now as the table was found in ENABLING state but not added to META the
EnableTableHandler
{code}
2013-04-03 18:33:03,850 FATAL org.apache.hadoop.hbase.master.HMaster:
Unhandled exception. Starting shutdown.
org.apache.hadoop.hbase.exceptions.TableNotFoundException: TestTable
at
org.apache.hadoop.hbase.master.handler.EnableTableHandler.prepare(EnableTableHandler.java:89)
at
org.apache.hadoop.hbase.master.AssignmentManager.recoverTableInEnablingState(AssignmentManager.java:2586)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:390)
at
[jira] [Commented] (HBASE-8314) HLogSplitter can retry to open a 0-length hlog file
[
https://issues.apache.org/jira/browse/HBASE-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627993#comment-13627993
]
Ted Yu commented on HBASE-8314:
---
Which hadoop version was used ?
I wonder if HBASE-7878 would make a difference.
HLogSplitter can retry to open a 0-length hlog file
---
Key: HBASE-8314
URL: https://issues.apache.org/jira/browse/HBASE-8314
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: region-server.log
In case a HLog file is of size 0, and it is under recovery, HLogSplitter will
fail to open it since it can get the file length, therefore, master can't
start.
{noformat}
java.io.IOException: Cannot obtain block length for LocatedBlock{...;
getBlockSize()=0; corrupt=false; offset=0; locs=[...]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:238)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:182)
at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:124)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:117)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1080)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8313) Add Bloom filter testing for HFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-8313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627997#comment-13627997 ] Hudson commented on HBASE-8313: --- Integrated in HBase-0.94 #953 (See [https://builds.apache.org/job/HBase-0.94/953/]) HBASE-8313 Add Bloom filter testing for HFileOutputFormat (Revision 1466418) Result = SUCCESS mbertozzi : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java Add Bloom filter testing for HFileOutputFormat -- Key: HBASE-8313 URL: https://issues.apache.org/jira/browse/HBASE-8313 Project: HBase Issue Type: Bug Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.94.7, 0.95.1 Attachments: HBASE-8313-94.patch, HBASE-8313-v0.patch HBASE-3776 added Bloom Filter Support to the HFileOutputFormat, but there's no test to verify that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7605) TestMiniClusterLoadSequential fails in trunk build on hadoop 2
[
https://issues.apache.org/jira/browse/HBASE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13627999#comment-13627999
]
Jonathan Hsieh commented on HBASE-7605:
---
tl;dr Test is correct but there is some regression when using hadoop2. Going
to reduce the size of the test so that it finishes in a reasonable amount of
time and passes. Would like to close with patch that makes it work but will
file jira to investigate why the big h1 vs h2 perf difference.
On my test runs against hadoop2, I get failures due to timeouts with
TestMiniClusterLoadParallel and TestMiniClusterLoadSequential.
With hadoop1 runs from the ec2 test trunk on hadoop2 instance [1], each of
these individual cases take this long to run.
*Sequential
{code}
Test name Duration Status
loadTest[0] 1 min 11 secPassed
loadTest[1] 48 sec Passed
loadTest[2] 22 sec Passed
loadTest[3] 27 sec Passed
{code}
*Parallel
{code}
Test name Duration Status
loadTest[0] 1 min 18 secPassed
loadTest[1] 51 sec Passed
loadTest[2] 19 sec Passed
loadTest[3] 20 sec Passed
{code}
I ran then locally on my box with hadoop1 profile an got this:
With hadoop2 I did some experiments on my local machine, bumping up Sequential
from 180s to 360s timeouts per test. They passed, but barely before timing
out. For some reason they are 2-3x slower. (Not good)
{code}
testcase time=348.761
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[0]/
testcase time=343.577
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[1]/
testcase time=46.397
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[2]/
testcase time=45.845
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[3]/
...
testcase time=358.088
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[0]/
testcase time=357.654
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[1]/
testcase time=61.306
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[2]/
testcase time=61.263
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[3]/
{code}
Instead of bumping timeout up, I'm going to change the tests so that it does an
order of magnitude less work, and finishes on the order of 10-30s instead of
100-300s.
Here are the hadoop2 vs hadoop1 results (bumping down from 1 keys to 1000
keys)
{code}
hadoop1:
testcase time=14.929
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[0]/
testcase time=12.747
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[1]/
testcase time=12.005
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[2]/
testcase time=11.663
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[3]/
testcase time=15.956
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[0]/
testcase time=14.321
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[1]/
testcase time=11.541
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[2]/
testcase time=11.588
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[3]/
hadoop2:
testcase time=43.703
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[0]/
testcase time=41.542
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[1]/
testcase time=10.711
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[2]/
testcase time=8.801
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel
name=loadTest[3]/
testcase time=45.986
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[0]/
testcase time=42.037
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[1]/
testcase time=12.56
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[2]/
testcase time=12.01
classname=org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
name=loadTest[3]/
{code}
[1]http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/53/
TestMiniClusterLoadSequential fails in trunk build on hadoop 2
--
Key: HBASE-7605
URL: https://issues.apache.org/jira/browse/HBASE-7605
Project: HBase
Issue Type: Sub-task
Reporter: Ted Yu
Priority: Critical
Fix For: 0.95.1
From HBase-TRUNK-on-Hadoop-2.0.0 #354:
[jira] [Assigned] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reassigned HBASE-8316: - Assignee: Lars Hofhansl (was: Ted Yu) JoinedHeap for non essential column families should reseek instead of seek -- Key: HBASE-8316 URL: https://issues.apache.org/jira/browse/HBASE-8316 Project: HBase Issue Type: Sub-task Components: Filters, Performance, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.98.0, 0.94.7, 0.95.1 Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt This was raised by the Phoenix team. During a profiling session we noticed that catching the joinedHeap up to the current rows via seek causes a performance regression, which makes the joinedHeap only efficient when either a high or low percentage is matched by the filter. (High is fine, because the joinedHeap will not get behind as often and does not need to be caught up, low is fine, because the seek isn't happening frequently). In our tests we found that the solution is quite simple: Replace seek with reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7605) TestMiniClusterLoadSequential fails in trunk build on hadoop 2
[ https://issues.apache.org/jira/browse/HBASE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-7605: -- Fix Version/s: 0.98.0 Status: Patch Available (was: Open) TestMiniClusterLoadSequential fails in trunk build on hadoop 2 -- Key: HBASE-7605 URL: https://issues.apache.org/jira/browse/HBASE-7605 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Priority: Critical Fix For: 0.98.0, 0.95.1 Attachments: hbase-7605.patch From HBase-TRUNK-on-Hadoop-2.0.0 #354: loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[2](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[3](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7605) TestMiniClusterLoadSequential fails in trunk build on hadoop 2
[ https://issues.apache.org/jira/browse/HBASE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reassigned HBASE-7605: - Assignee: Jonathan Hsieh TestMiniClusterLoadSequential fails in trunk build on hadoop 2 -- Key: HBASE-7605 URL: https://issues.apache.org/jira/browse/HBASE-7605 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Assignee: Jonathan Hsieh Priority: Critical Fix For: 0.98.0, 0.95.1 Attachments: hbase-7605.patch From HBase-TRUNK-on-Hadoop-2.0.0 #354: loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[2](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[3](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7605) TestMiniClusterLoadSequential fails in trunk build on hadoop 2
[ https://issues.apache.org/jira/browse/HBASE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-7605: -- Component/s: test hadoop2 TestMiniClusterLoadSequential fails in trunk build on hadoop 2 -- Key: HBASE-7605 URL: https://issues.apache.org/jira/browse/HBASE-7605 Project: HBase Issue Type: Sub-task Components: hadoop2, test Reporter: Ted Yu Assignee: Jonathan Hsieh Priority: Critical Fix For: 0.98.0, 0.95.1 Attachments: hbase-7605.patch From HBase-TRUNK-on-Hadoop-2.0.0 #354: loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[2](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[3](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7605) TestMiniClusterLoadSequential fails in trunk build on hadoop 2
[ https://issues.apache.org/jira/browse/HBASE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-7605: -- Attachment: hbase-7605.patch TestMiniClusterLoadSequential fails in trunk build on hadoop 2 -- Key: HBASE-7605 URL: https://issues.apache.org/jira/browse/HBASE-7605 Project: HBase Issue Type: Sub-task Reporter: Ted Yu Priority: Critical Fix For: 0.95.1 Attachments: hbase-7605.patch From HBase-TRUNK-on-Hadoop-2.0.0 #354: loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[2](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds loadTest[3](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test timed out after 12 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628002#comment-13628002 ] Ted Yu commented on HBASE-1936: --- From https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console , looks like there was compilation error (on hadoop 2.0). ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Jimmy Xiang Labels: noob Fix For: 0.98.0, 0.95.1 Attachments: cp_from_hdfs.patch, HBASE-1936-trunk(forReview).patch, trunk-1936.patch, trunk-1936_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8320) test-patch.sh doesn't post back compilation error
Ted Yu created HBASE-8320:
-
Summary: test-patch.sh doesn't post back compilation error
Key: HBASE-8320
URL: https://issues.apache.org/jira/browse/HBASE-8320
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
One example was:
https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console
{code}
if [[ $? != 0 ]] ; then
JIRA_COMMENT=$JIRA_COMMENT
{color:red}-1 hadoop2.0{color}. The patch failed to compile against the
hadoop 2.0 profile.
cleanupAndExit 1
fi
{code}
cleanupAndExit doesn't post the comment.
I think the correct call should be:
{code}
submitJiraComment 1
cleanupAndExit 1
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8320) test-patch.sh doesn't post back compilation error
[
https://issues.apache.org/jira/browse/HBASE-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8320:
--
Attachment: 8320.txt
test-patch.sh doesn't post back compilation error
-
Key: HBASE-8320
URL: https://issues.apache.org/jira/browse/HBASE-8320
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 8320.txt
One example was:
https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console
{code}
if [[ $? != 0 ]] ; then
JIRA_COMMENT=$JIRA_COMMENT
{color:red}-1 hadoop2.0{color}. The patch failed to compile against the
hadoop 2.0 profile.
cleanupAndExit 1
fi
{code}
cleanupAndExit doesn't post the comment.
I think the correct call should be:
{code}
submitJiraComment 1
cleanupAndExit 1
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8320) test-patch.sh doesn't post back compilation error
[
https://issues.apache.org/jira/browse/HBASE-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8320:
--
Status: Patch Available (was: Open)
test-patch.sh doesn't post back compilation error
-
Key: HBASE-8320
URL: https://issues.apache.org/jira/browse/HBASE-8320
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 8320.txt
One example was:
https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console
{code}
if [[ $? != 0 ]] ; then
JIRA_COMMENT=$JIRA_COMMENT
{color:red}-1 hadoop2.0{color}. The patch failed to compile against the
hadoop 2.0 profile.
cleanupAndExit 1
fi
{code}
cleanupAndExit doesn't post the comment.
I think the correct call should be:
{code}
submitJiraComment 1
cleanupAndExit 1
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7824) Improve master start up time when there is log splitting work
[ https://issues.apache.org/jira/browse/HBASE-7824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628024#comment-13628024 ] Jeffrey Zhong commented on HBASE-7824: -- Many thanks to Ram, Chunhui and Rajesh for the latest reviews! [~lhofhansl] Are you all right to check the patch v10 in? Thanks. Improve master start up time when there is log splitting work - Key: HBASE-7824 URL: https://issues.apache.org/jira/browse/HBASE-7824 Project: HBase Issue Type: Bug Components: master Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.94.8 Attachments: hbase-7824.patch, hbase-7824-v10.patch, hbase-7824_v2.patch, hbase-7824_v3.patch, hbase-7824-v7.patch, hbase-7824-v8.patch, hbase-7824-v9.patch When there is log split work going on, master start up waits till all log split work completes even though the log split has nothing to do with meta region servers. It's a bad behavior considering a master node can run when log split is happening while its start up is blocking by log split work. Since master is kind of single point of failure, we should start it ASAP. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7658) grant with an empty string as permission should throw an exception
[
https://issues.apache.org/jira/browse/HBASE-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628025#comment-13628025
]
Hadoop QA commented on HBASE-7658:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578030/HBASE-7658-v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5244//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5244//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5244//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5244//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5244//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5244//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5244//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5244//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5244//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5244//console
This message is automatically generated.
grant with an empty string as permission should throw an exception
--
Key: HBASE-7658
URL: https://issues.apache.org/jira/browse/HBASE-7658
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.2
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Attachments: HBASE-7658-v0.patch, HBASE-7658-v1.patch
If someone specify an empty permission
{code}grant 'user', ''{code}
AccessControlLists.addUserPermission() output a log message and doesn't
change the permission, but the user doesn't know about it.
{code}
if ((actions == null) || (actions.length == 0)) {
LOG.warn(No actions associated with user
'+Bytes.toString(userPerm.getUser())+');
return;
}
{code}
I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628026#comment-13628026 ] Jimmy Xiang commented on HBASE-1936: hbase-common must be installed so that hbase-client can be compiled. How does the pre-commit handle this? ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Jimmy Xiang Labels: noob Fix For: 0.98.0, 0.95.1 Attachments: cp_from_hdfs.patch, HBASE-1936-trunk(forReview).patch, trunk-1936.patch, trunk-1936_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628029#comment-13628029 ] Ted Yu commented on HBASE-1936: --- protobuf has been upgraded to 2.5.0 Looks like patch needs update: 1 out of 6 hunks FAILED -- saving rejects to file hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java.rej ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Jimmy Xiang Labels: noob Fix For: 0.98.0, 0.95.1 Attachments: cp_from_hdfs.patch, HBASE-1936-trunk(forReview).patch, trunk-1936.patch, trunk-1936_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8314) HLogSplitter can retry to open a 0-length hlog file
[
https://issues.apache.org/jira/browse/HBASE-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628031#comment-13628031
]
Jimmy Xiang commented on HBASE-8314:
Hadoop 0.20? Yes, HBASE-7878 helps some. We can try to re-open the hlog file
too.
HLogSplitter can retry to open a 0-length hlog file
---
Key: HBASE-8314
URL: https://issues.apache.org/jira/browse/HBASE-8314
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: region-server.log
In case a HLog file is of size 0, and it is under recovery, HLogSplitter will
fail to open it since it can get the file length, therefore, master can't
start.
{noformat}
java.io.IOException: Cannot obtain block length for LocatedBlock{...;
getBlockSize()=0; corrupt=false; offset=0; locs=[...]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:238)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:182)
at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:124)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:117)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1080)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7606) TestJoinedScanners fails in trunk build on hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh reassigned HBASE-7606:
-
Assignee: Jonathan Hsieh
TestJoinedScanners fails in trunk build on hadoop 2.0
-
Key: HBASE-7606
URL: https://issues.apache.org/jira/browse/HBASE-7606
Project: HBase
Issue Type: Sub-task
Reporter: Ted Yu
Assignee: Jonathan Hsieh
From
https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/353/testReport/junit/org.apache.hadoop.hbase.regionserver/TestJoinedScanners/testJoinedScanners/
:
{code}
2013-01-17 13:41:18,113 WARN
[RegionServer:2;juno.apache.org,44920,1358430022974.cacheFlusher]
hdfs.DFSInputStream(664): DFS Read
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
BP-624872226-67.195.138.61-1358430016098:blk_2914319395625853770_1224
file=/user/jenkins/hbase/TestJoinedScanners/4742bbe27e31aed5dfafd8b1e6dede03/.tmp/4ca392f6a0154df79cfafd3448f2c8aa
at
org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:734)
at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:448)
at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:645)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:689)
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:428)
at
org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:555)
at
org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:599)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1294)
at
org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:525)
at
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:628)
at
org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1259)
at
org.apache.hadoop.hbase.regionserver.HStore.commitFile(HStore.java:844)
at
org.apache.hadoop.hbase.regionserver.HStore.access$400(HStore.java:108)
at
org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:1841)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1606)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1504)
at
org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1445)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:410)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:384)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:247)
at java.lang.Thread.run(Thread.java:662)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7606) TestJoinedScanners fails in trunk build on hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628039#comment-13628039
]
Jonathan Hsieh commented on HBASE-7606:
---
Yikes! hadoop1 version of this used 10k rows and the test took 9+ minutes to
complete. [1]
{code}
Test name Duration Status
testJoinedScanners 9 min 8 sec Passed
{code}
I've notched it down to 1k rows and the test completes, it passes for me
locally for both h1 and h2 profiles on the order of a minute. Hadoop2 has a
2.5x regression, similar to HBASE-7605.
{code}
hadoop1:
testcase time=30.496
classname=org.apache.hadoop.hbase.regionserver.TestJoinedScanners
name=testJoinedScanners/
hadoop2:
testcase time=80.085
classname=org.apache.hadoop.hbase.regionserver.TestJoinedScanners
name=testJoinedScanners/
{code}
[1]
http://54.241.6.143/job/HBase-TRUNK/org.apache.hbase$hbase-server/119/testReport/org.apache.hadoop.hbase.regionserver/TestJoinedScanners/
TestJoinedScanners fails in trunk build on hadoop 2.0
-
Key: HBASE-7606
URL: https://issues.apache.org/jira/browse/HBASE-7606
Project: HBase
Issue Type: Sub-task
Reporter: Ted Yu
Assignee: Jonathan Hsieh
Attachments: hbase-7606.patch
From
https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/353/testReport/junit/org.apache.hadoop.hbase.regionserver/TestJoinedScanners/testJoinedScanners/
:
{code}
2013-01-17 13:41:18,113 WARN
[RegionServer:2;juno.apache.org,44920,1358430022974.cacheFlusher]
hdfs.DFSInputStream(664): DFS Read
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
BP-624872226-67.195.138.61-1358430016098:blk_2914319395625853770_1224
file=/user/jenkins/hbase/TestJoinedScanners/4742bbe27e31aed5dfafd8b1e6dede03/.tmp/4ca392f6a0154df79cfafd3448f2c8aa
at
org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:734)
at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:448)
at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:645)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:689)
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:428)
at
org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:555)
at
org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:599)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1294)
at
org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:525)
at
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:628)
at
org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1259)
at
org.apache.hadoop.hbase.regionserver.HStore.commitFile(HStore.java:844)
at
org.apache.hadoop.hbase.regionserver.HStore.access$400(HStore.java:108)
at
org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:1841)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1606)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1504)
at
org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1445)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:410)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:384)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:247)
at java.lang.Thread.run(Thread.java:662)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7606) TestJoinedScanners fails in trunk build on hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-7606:
--
Attachment: hbase-7606.patch
TestJoinedScanners fails in trunk build on hadoop 2.0
-
Key: HBASE-7606
URL: https://issues.apache.org/jira/browse/HBASE-7606
Project: HBase
Issue Type: Sub-task
Reporter: Ted Yu
Assignee: Jonathan Hsieh
Attachments: hbase-7606.patch
From
https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/353/testReport/junit/org.apache.hadoop.hbase.regionserver/TestJoinedScanners/testJoinedScanners/
:
{code}
2013-01-17 13:41:18,113 WARN
[RegionServer:2;juno.apache.org,44920,1358430022974.cacheFlusher]
hdfs.DFSInputStream(664): DFS Read
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
BP-624872226-67.195.138.61-1358430016098:blk_2914319395625853770_1224
file=/user/jenkins/hbase/TestJoinedScanners/4742bbe27e31aed5dfafd8b1e6dede03/.tmp/4ca392f6a0154df79cfafd3448f2c8aa
at
org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:734)
at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:448)
at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:645)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:689)
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:428)
at
org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:555)
at
org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:599)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1294)
at
org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:525)
at
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:628)
at
org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1259)
at
org.apache.hadoop.hbase.regionserver.HStore.commitFile(HStore.java:844)
at
org.apache.hadoop.hbase.regionserver.HStore.access$400(HStore.java:108)
at
org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:1841)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1606)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1504)
at
org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1445)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:410)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:384)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:247)
at java.lang.Thread.run(Thread.java:662)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7606) TestJoinedScanners fails in trunk build on hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-7606:
--
Fix Version/s: 0.95.1
0.98.0
Status: Patch Available (was: Reopened)
TestJoinedScanners fails in trunk build on hadoop 2.0
-
Key: HBASE-7606
URL: https://issues.apache.org/jira/browse/HBASE-7606
Project: HBase
Issue Type: Sub-task
Reporter: Ted Yu
Assignee: Jonathan Hsieh
Fix For: 0.98.0, 0.95.1
Attachments: hbase-7606.patch
From
https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/353/testReport/junit/org.apache.hadoop.hbase.regionserver/TestJoinedScanners/testJoinedScanners/
:
{code}
2013-01-17 13:41:18,113 WARN
[RegionServer:2;juno.apache.org,44920,1358430022974.cacheFlusher]
hdfs.DFSInputStream(664): DFS Read
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
BP-624872226-67.195.138.61-1358430016098:blk_2914319395625853770_1224
file=/user/jenkins/hbase/TestJoinedScanners/4742bbe27e31aed5dfafd8b1e6dede03/.tmp/4ca392f6a0154df79cfafd3448f2c8aa
at
org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:734)
at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:448)
at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:645)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:689)
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:428)
at
org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:555)
at
org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:599)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1294)
at
org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:525)
at
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:628)
at
org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1259)
at
org.apache.hadoop.hbase.regionserver.HStore.commitFile(HStore.java:844)
at
org.apache.hadoop.hbase.regionserver.HStore.access$400(HStore.java:108)
at
org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:1841)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1606)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1504)
at
org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1445)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:410)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:384)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:247)
at java.lang.Thread.run(Thread.java:662)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7606) TestJoinedScanners fails in trunk build on hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-7606:
--
Component/s: test
hadoop2
TestJoinedScanners fails in trunk build on hadoop 2.0
-
Key: HBASE-7606
URL: https://issues.apache.org/jira/browse/HBASE-7606
Project: HBase
Issue Type: Sub-task
Components: hadoop2, test
Reporter: Ted Yu
Assignee: Jonathan Hsieh
Fix For: 0.98.0, 0.95.1
Attachments: hbase-7606.patch
From
https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/353/testReport/junit/org.apache.hadoop.hbase.regionserver/TestJoinedScanners/testJoinedScanners/
:
{code}
2013-01-17 13:41:18,113 WARN
[RegionServer:2;juno.apache.org,44920,1358430022974.cacheFlusher]
hdfs.DFSInputStream(664): DFS Read
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
BP-624872226-67.195.138.61-1358430016098:blk_2914319395625853770_1224
file=/user/jenkins/hbase/TestJoinedScanners/4742bbe27e31aed5dfafd8b1e6dede03/.tmp/4ca392f6a0154df79cfafd3448f2c8aa
at
org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:734)
at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:448)
at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:645)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:689)
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:428)
at
org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:555)
at
org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:599)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1294)
at
org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:525)
at
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:628)
at
org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1259)
at
org.apache.hadoop.hbase.regionserver.HStore.commitFile(HStore.java:844)
at
org.apache.hadoop.hbase.regionserver.HStore.access$400(HStore.java:108)
at
org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.commit(HStore.java:1841)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1606)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1504)
at
org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1445)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:410)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:384)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:247)
at java.lang.Thread.run(Thread.java:662)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8317) Seek returns wrong result with PREFIX_TREE Encoding
[ https://issues.apache.org/jira/browse/HBASE-8317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628044#comment-13628044 ] Matt Corgan commented on HBASE-8317: Thanks for testing out the PREFIX_TREE guys. I will try to look into this one tonight. Let me know if you find anything else before then. Seek returns wrong result with PREFIX_TREE Encoding --- Key: HBASE-8317 URL: https://issues.apache.org/jira/browse/HBASE-8317 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: chunhui shen Assignee: chunhui shen Attachments: hbase-trunk-8317.patch TestPrefixTreeEncoding#testSeekWithFixedData from the patch could reproduce the bug. An example of the bug case: Suppose the following rows: 1.row3/c1:q1/ 2.row3/c1:q2/ 3.row3/c1:q3/ 4.row4/c1:q1/ 5.row4/c1:q2/ After seeking the row 'row30', the expected peek KV is row4/c1:q1/, but actual is row3/c1:q1/. I just fix this bug case in the patch, Maybe we can do more for other potential problems if anyone is familiar with the code of PREFIX_TREE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7658) grant with an empty string as permission should throw an exception
[
https://issues.apache.org/jira/browse/HBASE-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-7658:
---
Resolution: Fixed
Fix Version/s: 0.95.1
Status: Resolved (was: Patch Available)
grant with an empty string as permission should throw an exception
--
Key: HBASE-7658
URL: https://issues.apache.org/jira/browse/HBASE-7658
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.2
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 0.95.1
Attachments: HBASE-7658-v0.patch, HBASE-7658-v1.patch
If someone specify an empty permission
{code}grant 'user', ''{code}
AccessControlLists.addUserPermission() output a log message and doesn't
change the permission, but the user doesn't know about it.
{code}
if ((actions == null) || (actions.length == 0)) {
LOG.warn(No actions associated with user
'+Bytes.toString(userPerm.getUser())+');
return;
}
{code}
I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-7658) grant with an empty string as permission should throw an exception
[
https://issues.apache.org/jira/browse/HBASE-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reopened HBASE-7658:
---
What about 0.94?
grant with an empty string as permission should throw an exception
--
Key: HBASE-7658
URL: https://issues.apache.org/jira/browse/HBASE-7658
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.2
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 0.95.1
Attachments: HBASE-7658-v0.patch, HBASE-7658-v1.patch
If someone specify an empty permission
{code}grant 'user', ''{code}
AccessControlLists.addUserPermission() output a log message and doesn't
change the permission, but the user doesn't know about it.
{code}
if ((actions == null) || (actions.length == 0)) {
LOG.warn(No actions associated with user
'+Bytes.toString(userPerm.getUser())+');
return;
}
{code}
I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-1936: --- Status: Open (was: Patch Available) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Jimmy Xiang Labels: noob Fix For: 0.98.0, 0.95.1 Attachments: cp_from_hdfs.patch, HBASE-1936-trunk(forReview).patch, trunk-1936.patch, trunk-1936_v2.1.patch, trunk-1936_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-1936: --- Attachment: trunk-1936_v2.1.patch ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Jimmy Xiang Labels: noob Fix For: 0.98.0, 0.95.1 Attachments: cp_from_hdfs.patch, HBASE-1936-trunk(forReview).patch, trunk-1936.patch, trunk-1936_v2.1.patch, trunk-1936_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-1936: --- Status: Patch Available (was: Open) Rebased to trunk latest. ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Jimmy Xiang Labels: noob Fix For: 0.98.0, 0.95.1 Attachments: cp_from_hdfs.patch, HBASE-1936-trunk(forReview).patch, trunk-1936.patch, trunk-1936_v2.1.patch, trunk-1936_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7605) TestMiniClusterLoadSequential fails in trunk build on hadoop 2
[
https://issues.apache.org/jira/browse/HBASE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628084#comment-13628084
]
Hadoop QA commented on HBASE-7605:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578035/hbase-7605.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5245//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5245//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5245//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5245//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5245//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5245//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5245//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5245//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5245//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5245//console
This message is automatically generated.
TestMiniClusterLoadSequential fails in trunk build on hadoop 2
--
Key: HBASE-7605
URL: https://issues.apache.org/jira/browse/HBASE-7605
Project: HBase
Issue Type: Sub-task
Components: hadoop2, test
Reporter: Ted Yu
Assignee: Jonathan Hsieh
Priority: Critical
Fix For: 0.98.0, 0.95.1
Attachments: hbase-7605.patch
From HBase-TRUNK-on-Hadoop-2.0.0 #354:
loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential):
test timed out after 12 milliseconds
loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential):
test timed out after 12 milliseconds
loadTest[2](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential):
test timed out after 12 milliseconds
loadTest[3](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential):
test timed out after 12 milliseconds
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7247) Assignment performances decreased by 50% because of regionserver.OpenRegionHandler#tickleOpening
[ https://issues.apache.org/jira/browse/HBASE-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628089#comment-13628089 ] stack commented on HBASE-7247: -- Thanks for explanation. +1 on commit to trunk and 0.95. Assignment performances decreased by 50% because of regionserver.OpenRegionHandler#tickleOpening Key: HBASE-7247 URL: https://issues.apache.org/jira/browse/HBASE-7247 Project: HBase Issue Type: Improvement Components: master, Region Assignment, regionserver Affects Versions: 0.95.2 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.98.0, 0.95.1 Attachments: 7247.v1.patch, 7247.v2.patch The regionserver.OpenRegionHandler#tickleOpening updates the region znode as Do this so master doesn't timeout this region-in-transition.. However, on the usual test, this makes the assignment time of 1500 regions goes from 70s to 100s, that is, we're 50% slower because of this. More generally, ZooKeper commits to disk all the data update, and this takes time. Using it to provide a keep alive seems overkill. At the very list, it could be made asynchronous. I'm not sure how necessary these updates are required (I need to go deeper in the internal, feedback welcome), but it seems very important to optimize this... The trival fix would be to make this optional. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8320) test-patch.sh doesn't post back compilation error
[
https://issues.apache.org/jira/browse/HBASE-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628095#comment-13628095
]
Hadoop QA commented on HBASE-8320:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578037/8320.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5246//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5246//console
This message is automatically generated.
test-patch.sh doesn't post back compilation error
-
Key: HBASE-8320
URL: https://issues.apache.org/jira/browse/HBASE-8320
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 8320.txt
One example was:
https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console
{code}
if [[ $? != 0 ]] ; then
JIRA_COMMENT=$JIRA_COMMENT
{color:red}-1 hadoop2.0{color}. The patch failed to compile against the
hadoop 2.0 profile.
cleanupAndExit 1
fi
{code}
cleanupAndExit doesn't post the comment.
I think the correct call should be:
{code}
submitJiraComment 1
cleanupAndExit 1
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7658) grant with an empty string as permission should throw an exception
[
https://issues.apache.org/jira/browse/HBASE-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628100#comment-13628100
]
Andrew Purtell commented on HBASE-7658:
---
+1, thanks. Ping [~lhofhansl]
grant with an empty string as permission should throw an exception
--
Key: HBASE-7658
URL: https://issues.apache.org/jira/browse/HBASE-7658
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.2
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 0.95.1
Attachments: HBASE-7658-0.94.patch, HBASE-7658-v0.patch,
HBASE-7658-v1.patch
If someone specify an empty permission
{code}grant 'user', ''{code}
AccessControlLists.addUserPermission() output a log message and doesn't
change the permission, but the user doesn't know about it.
{code}
if ((actions == null) || (actions.length == 0)) {
LOG.warn(No actions associated with user
'+Bytes.toString(userPerm.getUser())+');
return;
}
{code}
I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7658) grant with an empty string as permission should throw an exception
[
https://issues.apache.org/jira/browse/HBASE-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-7658:
---
Attachment: HBASE-7658-0.94.patch
94 patch, is the same as trunk. If no objection I'll backport it
grant with an empty string as permission should throw an exception
--
Key: HBASE-7658
URL: https://issues.apache.org/jira/browse/HBASE-7658
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.2
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Trivial
Fix For: 0.95.1
Attachments: HBASE-7658-0.94.patch, HBASE-7658-v0.patch,
HBASE-7658-v1.patch
If someone specify an empty permission
{code}grant 'user', ''{code}
AccessControlLists.addUserPermission() output a log message and doesn't
change the permission, but the user doesn't know about it.
{code}
if ((actions == null) || (actions.length == 0)) {
LOG.warn(No actions associated with user
'+Bytes.toString(userPerm.getUser())+');
return;
}
{code}
I think we should throw an exception instead of just logging.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628129#comment-13628129 ] Hudson commented on HBASE-8165: --- Integrated in HBase-TRUNK #4050 (See [https://builds.apache.org/job/HBase-TRUNK/4050/]) HBASE-8165 Update our protobuf to 2.5 from 2.4.1 (Revision 1466556) Result = SUCCESS stack : Files : * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/ServerName.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AccessControlProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AggregateProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AuthenticationProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterIdProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterStatusProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ComparatorProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ErrorHandlingProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FSProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FilterProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HFileProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/LoadBalancerProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MapReduceProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterMonitorProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MultiRowMutation.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MultiRowMutationProcessorProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionServerStatusProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RowProcessorProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/SecureBulkLoadProtos.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/Tracing.java * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/MasterAdmin.proto * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/CellMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/CellSetMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/ColumnSchemaMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/ScannerMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/StorageClusterStatusMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/TableInfoMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/TableListMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/TableSchemaMessage.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/rest/protobuf/generated/VersionMessage.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/protobuf/generated/ColumnAggregationProtos.java *
[jira] [Updated] (HBASE-8308) Lower the number of expected regions in TestTableLockManager#testTableReadLock
[ https://issues.apache.org/jira/browse/HBASE-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-8308: -- Resolution: Fixed Status: Resolved (was: Patch Available) Looks like we have several green builds. Lower the number of expected regions in TestTableLockManager#testTableReadLock -- Key: HBASE-8308 URL: https://issues.apache.org/jira/browse/HBASE-8308 Project: HBase Issue Type: Test Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.98.0, 0.95.1 Attachments: 8308.txt TestTableLockManager#testTableReadLock finishes when there're 10 regions from splitting action. This number may be a bit high. On a loaded build machine, the test would timeout. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7606) TestJoinedScanners fails in trunk build on hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628144#comment-13628144
]
Hadoop QA commented on HBASE-7606:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578040/hbase-7606.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5247//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5247//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5247//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5247//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5247//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5247//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5247//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5247//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5247//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5247//console
This message is automatically generated.
TestJoinedScanners fails in trunk build on hadoop 2.0
-
Key: HBASE-7606
URL: https://issues.apache.org/jira/browse/HBASE-7606
Project: HBase
Issue Type: Sub-task
Components: hadoop2, test
Reporter: Ted Yu
Assignee: Jonathan Hsieh
Fix For: 0.98.0, 0.95.1
Attachments: hbase-7606.patch
From
https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/353/testReport/junit/org.apache.hadoop.hbase.regionserver/TestJoinedScanners/testJoinedScanners/
:
{code}
2013-01-17 13:41:18,113 WARN
[RegionServer:2;juno.apache.org,44920,1358430022974.cacheFlusher]
hdfs.DFSInputStream(664): DFS Read
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
BP-624872226-67.195.138.61-1358430016098:blk_2914319395625853770_1224
file=/user/jenkins/hbase/TestJoinedScanners/4742bbe27e31aed5dfafd8b1e6dede03/.tmp/4ca392f6a0154df79cfafd3448f2c8aa
at
org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:734)
at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:448)
at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:645)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:689)
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:428)
at
org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:555)
at
org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:599)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1294)
at
org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:525)
at
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:628)
at
org.apache.hadoop.hbase.regionserver.HStore.validateStoreFile(HStore.java:1259)
at
[jira] [Commented] (HBASE-8256) Add category Flaky for tests which are flaky
[ https://issues.apache.org/jira/browse/HBASE-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628155#comment-13628155 ] Nicolas Liochon commented on HBASE-8256: Likely. And may be fixed in the latest versions (the ones I can't make working). Add category Flaky for tests which are flaky Key: HBASE-8256 URL: https://issues.apache.org/jira/browse/HBASE-8256 Project: HBase Issue Type: Bug Components: build, test Affects Versions: 0.98.0, 0.95.1 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Attachments: trunk-8256_v0.patch To make the Jenkin build more useful, it is good to keep it blue/green. We can mark those flaky tests flaky, and don't run them by default. However, people can still run them. We can also set up a Jekin build just for those flaky tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8320) test-patch.sh doesn't post back compilation error
[
https://issues.apache.org/jira/browse/HBASE-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628182#comment-13628182
]
Sergey Shelukhin commented on HBASE-8320:
-
looks reasonable
test-patch.sh doesn't post back compilation error
-
Key: HBASE-8320
URL: https://issues.apache.org/jira/browse/HBASE-8320
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 8320.txt
One example was:
https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console
{code}
if [[ $? != 0 ]] ; then
JIRA_COMMENT=$JIRA_COMMENT
{color:red}-1 hadoop2.0{color}. The patch failed to compile against the
hadoop 2.0 profile.
cleanupAndExit 1
fi
{code}
cleanupAndExit doesn't post the comment.
I think the correct call should be:
{code}
submitJiraComment 1
cleanupAndExit 1
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6295) Possible performance improvement in client batch operations: presplit and send in background
[
https://issues.apache.org/jira/browse/HBASE-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-6295:
---
Assignee: Nicolas Liochon
Possible performance improvement in client batch operations: presplit and
send in background
Key: HBASE-6295
URL: https://issues.apache.org/jira/browse/HBASE-6295
Project: HBase
Issue Type: Improvement
Components: Client, Performance
Affects Versions: 0.95.2
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Labels: noob
Attachments: 6295.v1.patch
today batch algo is:
{noformat}
for Operation o: ListOp{
add o to todolist
if todolist maxsize or o last in list
split todolist per location
send split lists to region servers
clear todolist
wait
}
{noformat}
We could:
- create immediately the final object instead of an intermediate array
- split per location immediately
- instead of sending when the list as a whole is full, send it when there is
enough data for a single location
It would be:
{noformat}
for Operation o: ListOp{
get location
add o to todo location.todolist
if (location.todolist maxLocationSize)
send location.todolist to region server
clear location.todolist
// don't wait, continue the loop
}
send remaining
wait
{noformat}
It's not trivial to write if you add error management: retried list must be
shared with the operations added in the todolist. But it's doable.
It's interesting mainly for 'big' writes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6295) Possible performance improvement in client batch operations: presplit and send in background
[
https://issues.apache.org/jira/browse/HBASE-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-6295:
---
Attachment: 6295.v1.patch
Possible performance improvement in client batch operations: presplit and
send in background
Key: HBASE-6295
URL: https://issues.apache.org/jira/browse/HBASE-6295
Project: HBase
Issue Type: Improvement
Components: Client, Performance
Affects Versions: 0.95.2
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Labels: noob
Attachments: 6295.v1.patch
today batch algo is:
{noformat}
for Operation o: ListOp{
add o to todolist
if todolist maxsize or o last in list
split todolist per location
send split lists to region servers
clear todolist
wait
}
{noformat}
We could:
- create immediately the final object instead of an intermediate array
- split per location immediately
- instead of sending when the list as a whole is full, send it when there is
enough data for a single location
It would be:
{noformat}
for Operation o: ListOp{
get location
add o to todo location.todolist
if (location.todolist maxLocationSize)
send location.todolist to region server
clear location.todolist
// don't wait, continue the loop
}
send remaining
wait
{noformat}
It's not trivial to write if you add error management: retried list must be
shared with the operations added in the todolist. But it's doable.
It's interesting mainly for 'big' writes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6295) Possible performance improvement in client batch operations: presplit and send in background
[
https://issues.apache.org/jira/browse/HBASE-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628203#comment-13628203
]
Nicolas Liochon commented on HBASE-6295:
It's still in a hacky way. Needs some work be be clean. I need to write some
unit tests, I'm sure this implementation is still buggy.
What I would like to keep is the logic I've finally implemented:
- when we send the buffered writes to the regionservers, we don't stop except
if we have to, i.e: we have too much tasks in progress or we ran out of retries.
- we don't have this terrible Object[] to send back the results. Actually, in
most methods, they are never used anyway.
- we don't keep the buffer for failed operation. In other words,
clearBufferOnFail would always be true (it's already the default).
If it works, it means we won't be slow down by the slowest region server
anymore.
Possible performance improvement in client batch operations: presplit and
send in background
Key: HBASE-6295
URL: https://issues.apache.org/jira/browse/HBASE-6295
Project: HBase
Issue Type: Improvement
Components: Client, Performance
Affects Versions: 0.95.2
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Labels: noob
Attachments: 6295.v1.patch
today batch algo is:
{noformat}
for Operation o: ListOp{
add o to todolist
if todolist maxsize or o last in list
split todolist per location
send split lists to region servers
clear todolist
wait
}
{noformat}
We could:
- create immediately the final object instead of an intermediate array
- split per location immediately
- instead of sending when the list as a whole is full, send it when there is
enough data for a single location
It would be:
{noformat}
for Operation o: ListOp{
get location
add o to todo location.todolist
if (location.todolist maxLocationSize)
send location.todolist to region server
clear location.todolist
// don't wait, continue the loop
}
send remaining
wait
{noformat}
It's not trivial to write if you add error management: retried list must be
shared with the operations added in the todolist. But it's doable.
It's interesting mainly for 'big' writes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6295) Possible performance improvement in client batch operations: presplit and send in background
[
https://issues.apache.org/jira/browse/HBASE-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Liochon updated HBASE-6295:
---
Status: Patch Available (was: Open)
Possible performance improvement in client batch operations: presplit and
send in background
Key: HBASE-6295
URL: https://issues.apache.org/jira/browse/HBASE-6295
Project: HBase
Issue Type: Improvement
Components: Client, Performance
Affects Versions: 0.95.2
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Labels: noob
Attachments: 6295.v1.patch
today batch algo is:
{noformat}
for Operation o: ListOp{
add o to todolist
if todolist maxsize or o last in list
split todolist per location
send split lists to region servers
clear todolist
wait
}
{noformat}
We could:
- create immediately the final object instead of an intermediate array
- split per location immediately
- instead of sending when the list as a whole is full, send it when there is
enough data for a single location
It would be:
{noformat}
for Operation o: ListOp{
get location
add o to todo location.todolist
if (location.todolist maxLocationSize)
send location.todolist to region server
clear location.todolist
// don't wait, continue the loop
}
send remaining
wait
{noformat}
It's not trivial to write if you add error management: retried list must be
shared with the operations added in the todolist. But it's doable.
It's interesting mainly for 'big' writes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8314) HLogSplitter can retry to open a 0-length hlog file
[
https://issues.apache.org/jira/browse/HBASE-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-8314:
---
Status: Patch Available (was: Open)
HLogSplitter can retry to open a 0-length hlog file
---
Key: HBASE-8314
URL: https://issues.apache.org/jira/browse/HBASE-8314
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: region-server.log, trunk-8314.patch
In case a HLog file is of size 0, and it is under recovery, HLogSplitter will
fail to open it since it can get the file length, therefore, master can't
start.
{noformat}
java.io.IOException: Cannot obtain block length for LocatedBlock{...;
getBlockSize()=0; corrupt=false; offset=0; locs=[...]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:238)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:182)
at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:124)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:117)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1080)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8314) HLogSplitter can retry to open a 0-length hlog file
[
https://issues.apache.org/jira/browse/HBASE-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-8314:
---
Attachment: trunk-8314.patch
HLogSplitter can retry to open a 0-length hlog file
---
Key: HBASE-8314
URL: https://issues.apache.org/jira/browse/HBASE-8314
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Attachments: region-server.log, trunk-8314.patch
In case a HLog file is of size 0, and it is under recovery, HLogSplitter will
fail to open it since it can get the file length, therefore, master can't
start.
{noformat}
java.io.IOException: Cannot obtain block length for LocatedBlock{...;
getBlockSize()=0; corrupt=false; offset=0; locs=[...]}
at
org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:238)
at
org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:182)
at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:124)
at org.apache.hadoop.hdfs.DFSInputStream.init(DFSInputStream.java:117)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1080)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8320) test-patch.sh should post back compilation error against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8320:
--
Summary: test-patch.sh should post back compilation error against hadoop
2.0 (was: test-patch.sh doesn't post back compilation error)
test-patch.sh should post back compilation error against hadoop 2.0
---
Key: HBASE-8320
URL: https://issues.apache.org/jira/browse/HBASE-8320
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 8320.txt
One example was:
https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console
{code}
if [[ $? != 0 ]] ; then
JIRA_COMMENT=$JIRA_COMMENT
{color:red}-1 hadoop2.0{color}. The patch failed to compile against the
hadoop 2.0 profile.
cleanupAndExit 1
fi
{code}
cleanupAndExit doesn't post the comment.
I think the correct call should be:
{code}
submitJiraComment 1
cleanupAndExit 1
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8320) test-patch.sh should post back compilation error against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628256#comment-13628256
]
Ted Yu commented on HBASE-8320:
---
Integrated to trunk.
Thanks for the review, Sergey.
test-patch.sh should post back compilation error against hadoop 2.0
---
Key: HBASE-8320
URL: https://issues.apache.org/jira/browse/HBASE-8320
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 8320.txt
One example was:
https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console
{code}
if [[ $? != 0 ]] ; then
JIRA_COMMENT=$JIRA_COMMENT
{color:red}-1 hadoop2.0{color}. The patch failed to compile against the
hadoop 2.0 profile.
cleanupAndExit 1
fi
{code}
cleanupAndExit doesn't post the comment.
I think the correct call should be:
{code}
submitJiraComment 1
cleanupAndExit 1
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8320) test-patch.sh should post back compilation error against hadoop 2.0
[
https://issues.apache.org/jira/browse/HBASE-8320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8320:
--
Fix Version/s: 0.98.0
Hadoop Flags: Reviewed
test-patch.sh should post back compilation error against hadoop 2.0
---
Key: HBASE-8320
URL: https://issues.apache.org/jira/browse/HBASE-8320
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 8320.txt
One example was:
https://builds.apache.org/job/PreCommit-HBASE-Build/5242/console
{code}
if [[ $? != 0 ]] ; then
JIRA_COMMENT=$JIRA_COMMENT
{color:red}-1 hadoop2.0{color}. The patch failed to compile against the
hadoop 2.0 profile.
cleanupAndExit 1
fi
{code}
cleanupAndExit doesn't post the comment.
I think the correct call should be:
{code}
submitJiraComment 1
cleanupAndExit 1
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8321) Log split worker should heartbeat to avoid timeout
Jimmy Xiang created HBASE-8321: -- Summary: Log split worker should heartbeat to avoid timeout Key: HBASE-8321 URL: https://issues.apache.org/jira/browse/HBASE-8321 Project: HBase Issue Type: Bug Components: wal Reporter: Jimmy Xiang Assignee: Jimmy Xiang Currently, hlog splitter could spend quite sometime to split a log in case any HDFS issue and recoverLease/retry opening is needed. If distributed log split manager times out the log worker, other log worker to take over will run into the same issue. Ideally, we should not need a timeout monitor. Since we have a timeout monitor for DSL now, the worker should heartbeat to avoid wrong/unneeded timeouts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8321) Log split worker should heartbeat to avoid timeout
[ https://issues.apache.org/jira/browse/HBASE-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628262#comment-13628262 ] Ted Yu commented on HBASE-8321: --- What is DSL ? Log split worker should heartbeat to avoid timeout -- Key: HBASE-8321 URL: https://issues.apache.org/jira/browse/HBASE-8321 Project: HBase Issue Type: Bug Components: wal Reporter: Jimmy Xiang Assignee: Jimmy Xiang Currently, hlog splitter could spend quite sometime to split a log in case any HDFS issue and recoverLease/retry opening is needed. If distributed log split manager times out the log worker, other log worker to take over will run into the same issue. Ideally, we should not need a timeout monitor. Since we have a timeout monitor for DSL now, the worker should heartbeat to avoid wrong/unneeded timeouts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8321) Log split worker should heartbeat to avoid timeout
[ https://issues.apache.org/jira/browse/HBASE-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628266#comment-13628266 ] Jimmy Xiang commented on HBASE-8321: Should be DLS, distributed log splitting. Log split worker should heartbeat to avoid timeout -- Key: HBASE-8321 URL: https://issues.apache.org/jira/browse/HBASE-8321 Project: HBase Issue Type: Bug Components: wal Reporter: Jimmy Xiang Assignee: Jimmy Xiang Currently, hlog splitter could spend quite sometime to split a log in case any HDFS issue and recoverLease/retry opening is needed. If distributed log split manager times out the log worker, other log worker to take over will run into the same issue. Ideally, we should not need a timeout monitor. Since we have a timeout monitor for DSL now, the worker should heartbeat to avoid wrong/unneeded timeouts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8165) Update our protobuf to 2.5 from 2.4.1
[ https://issues.apache.org/jira/browse/HBASE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628270#comment-13628270 ] Ted Yu commented on HBASE-8165: --- Looks like the test failures might be related to this JIRA: http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/107/testReport/ Update our protobuf to 2.5 from 2.4.1 - Key: HBASE-8165 URL: https://issues.apache.org/jira/browse/HBASE-8165 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.95.1 Attachments: 8165.txt, 8165v2.txt, 8165v3.txt, 8165v4.txt, 8165v5.txt Update to new 2.5 pb. Some speedups and a new PARSER idiom that bypasses making a builder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7704) migration tool that checks presence of HFile V1 files
[ https://issues.apache.org/jira/browse/HBASE-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13628273#comment-13628273 ] Ted Yu commented on HBASE-7704: --- Yeah. It was for HBASE-8288 migration tool that checks presence of HFile V1 files - Key: HBASE-7704 URL: https://issues.apache.org/jira/browse/HBASE-7704 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Himanshu Vashishtha Priority: Blocker Fix For: 0.95.1 Attachments: HBase-7704-v1.patch, HBase-7704-v2.patch, HBase-8288-v3.patch Below was Stack's comment from HBASE-7660: Regards the migration 'tool', or 'tool' to check for presence of v1 files, I imagine it as an addition to the hfile tool http://hbase.apache.org/book.html#hfile_tool2 The hfile tool already takes a bunch of args including printing out meta. We could add an option to print out version only – or return 1 if version 1 or some such – and then do a bit of code to just list all hfiles and run this script against each. Could MR it if too many files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8316) JoinedHeap for non essential column families should reseek instead of seek
[ https://issues.apache.org/jira/browse/HBASE-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8316: - Attachment: noencode.png FDencode.png Benchmark results (replacing seek with reseek, not requestSeek, but that will only make it better). The dotted line were the result before the change. JoinedHeap for non essential column families should reseek instead of seek -- Key: HBASE-8316 URL: https://issues.apache.org/jira/browse/HBASE-8316 Project: HBase Issue Type: Sub-task Components: Filters, Performance, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.98.0, 0.94.7, 0.95.1 Attachments: 8316-0.94.txt, 8316-trunk.txt, 8316-trunk.txt, FDencode.png, noencode.png This was raised by the Phoenix team. During a profiling session we noticed that catching the joinedHeap up to the current rows via seek causes a performance regression, which makes the joinedHeap only efficient when either a high or low percentage is matched by the filter. (High is fine, because the joinedHeap will not get behind as often and does not need to be caught up, low is fine, because the seek isn't happening frequently). In our tests we found that the solution is quite simple: Replace seek with reseek. Patch coming soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
