[jira] Updated: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-3234: - Fix Version/s: (was: 0.90.0) 0.90.1 OK. Chatting w/ Todd and Ryan, HDFS-724 is not in CDH. Removing HDFS-724, all hbase tests pass (Both for me and Ryan). HDFS-724 looks like critical fix but I'm going to go ahead and RC without it. There seems to be something up w/ the HDFS-724 that is in the append branch. While we're figuring it out, our first 0.90.0 RC can get an airing (Anyone can sink the RC if they disagree w/ above). My guess is that there'll at least be an RC2 and we can get any fix in then. I made an hadoop jar off the tip of branch-0.20-append that includes hdfs-895 but removes hdfs-724, signed it and put it up on a hand-built repo up on people.apache.org. The jar is named 0.20.3-append-r1034938-plusHDFS895-minusHDFS724. (@Gary -- thanks for the stack vs unknown user tidbit... I kinda picked up on it later but your clarification helped) hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932055#action_12932055 ] stack commented on HBASE-3234: -- Oh, just to say that Todd thinks it might be interaction between hdfs-724 and hdfs-895. He was going to try and look into it. hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932088#action_12932088 ] Todd Lipcon commented on HBASE-3234: bq. Oh, just to say that Todd thinks it might be interaction between hdfs-724 and hdfs-895. He was going to try and look into it. I found a bug on trunk HDFS-895 that was due to an issue interacting with HDFS-724, but according to people working on 20-append, this bug is present even with *just* 724 without 895. So I think it's a bad backport. hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2110) Move to ivy broke our finding stylesheet (Tables don't have blue lines around them any more).
[ https://issues.apache.org/jira/browse/HBASE-2110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932160#action_12932160 ] Lars Francke commented on HBASE-2110: - The problem has been fixed in Hadoop but as far as I can tell it's only in trunk so we need to keep those DTDs out of the JSPs until Hadoop 0.22 is released and used by HBase. Move to ivy broke our finding stylesheet (Tables don't have blue lines around them any more). - Key: HBASE-2110 URL: https://issues.apache.org/jira/browse/HBASE-2110 Project: HBase Issue Type: Bug Reporter: stack Move to ivy broke pulling in hbase.css from the static webapp; investigate. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932166#action_12932166 ] Jean-Daniel Cryans commented on HBASE-3234: --- So in our 0.90.0 RC email, we say that people can use the 0.20-append branch... but won't it be incompatible since it contains HDFS-724 which bumps the data transfer protocol? hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932188#action_12932188 ] Todd Lipcon commented on HBASE-3234: Ah, yes... but do you know of anyone who actually has run a cluster on 20-append? :) Probably not worth rebuilding RC, we can just tell people just kidding for now ;-) hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932190#action_12932190 ] Jean-Daniel Cryans commented on HBASE-3234: --- Thanks Todd, just wanted to confirm my understanding of the situation. hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HBASE-3236) Document API changes in 0.90 (from 0.20) -- the removal of deprecated classes
Document API changes in 0.90 (from 0.20) -- the removal of deprecated classes - Key: HBASE-3236 URL: https://issues.apache.org/jira/browse/HBASE-3236 Project: HBase Issue Type: Bug Reporter: stack Fix For: 0.90.1 jdcryans suggests a page in book where we list major changes in API -- the deprecated stuff removed in 0.90. Ted Yu started a list up in mailing list: I am trying to compile our code against 0.90 Result.getCellValue() is no longer available. Can you tell me the alternative ? .. HConstants is final class now instead of interface RowFilterInterface is gone org.apache.hadoop.hbase.io.Cell is gone org.apache.hadoop.hbase.io.RowResult is gone constructor HColumnDescriptor(byte[],int,java.lang.String,boolean,boolean,int,boolean) is gone Put.setTimeStamp() is gone org.apache.hadoop.hbase.filter.Filter has added getNextKeyHint(org.apache.hadoop.hbase.KeyValue) If you know the alternative to some of the old classes, please share. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3236) Document API changes in 0.90 (from 0.20) -- the removal of deprecated classes
[ https://issues.apache.org/jira/browse/HBASE-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932230#action_12932230 ] Todd Lipcon commented on HBASE-3236: In Hadoop-land we use jdiff to generate this. It's not too hard to set up, and it generates pretty nice output. Document API changes in 0.90 (from 0.20) -- the removal of deprecated classes - Key: HBASE-3236 URL: https://issues.apache.org/jira/browse/HBASE-3236 Project: HBase Issue Type: Bug Reporter: stack Fix For: 0.90.1 jdcryans suggests a page in book where we list major changes in API -- the deprecated stuff removed in 0.90. Ted Yu started a list up in mailing list: I am trying to compile our code against 0.90 Result.getCellValue() is no longer available. Can you tell me the alternative ? .. HConstants is final class now instead of interface RowFilterInterface is gone org.apache.hadoop.hbase.io.Cell is gone org.apache.hadoop.hbase.io.RowResult is gone constructor HColumnDescriptor(byte[],int,java.lang.String,boolean,boolean,int,boolean) is gone Put.setTimeStamp() is gone org.apache.hadoop.hbase.filter.Filter has added getNextKeyHint(org.apache.hadoop.hbase.KeyValue) If you know the alternative to some of the old classes, please share. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3236) Document API changes in 0.90 (from 0.20) -- the removal of deprecated classes
[ https://issues.apache.org/jira/browse/HBASE-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932237#action_12932237 ] stack commented on HBASE-3236: -- ok. let me try. will report back. Document API changes in 0.90 (from 0.20) -- the removal of deprecated classes - Key: HBASE-3236 URL: https://issues.apache.org/jira/browse/HBASE-3236 Project: HBase Issue Type: Bug Reporter: stack Fix For: 0.90.1 jdcryans suggests a page in book where we list major changes in API -- the deprecated stuff removed in 0.90. Ted Yu started a list up in mailing list: I am trying to compile our code against 0.90 Result.getCellValue() is no longer available. Can you tell me the alternative ? .. HConstants is final class now instead of interface RowFilterInterface is gone org.apache.hadoop.hbase.io.Cell is gone org.apache.hadoop.hbase.io.RowResult is gone constructor HColumnDescriptor(byte[],int,java.lang.String,boolean,boolean,int,boolean) is gone Put.setTimeStamp() is gone org.apache.hadoop.hbase.filter.Filter has added getNextKeyHint(org.apache.hadoop.hbase.KeyValue) If you know the alternative to some of the old classes, please share. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3236) Document API changes in 0.90 (from 0.20) -- the removal of deprecated classes
[ https://issues.apache.org/jira/browse/HBASE-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932239#action_12932239 ] Jean-Daniel Cryans commented on HBASE-3236: --- I was thinking of something a bit more beefy, like with examples and such. I'm trying to avoid what happened on the hadoop ML over the weekend re: 0.21 Document API changes in 0.90 (from 0.20) -- the removal of deprecated classes - Key: HBASE-3236 URL: https://issues.apache.org/jira/browse/HBASE-3236 Project: HBase Issue Type: Bug Reporter: stack Fix For: 0.90.1 jdcryans suggests a page in book where we list major changes in API -- the deprecated stuff removed in 0.90. Ted Yu started a list up in mailing list: I am trying to compile our code against 0.90 Result.getCellValue() is no longer available. Can you tell me the alternative ? .. HConstants is final class now instead of interface RowFilterInterface is gone org.apache.hadoop.hbase.io.Cell is gone org.apache.hadoop.hbase.io.RowResult is gone constructor HColumnDescriptor(byte[],int,java.lang.String,boolean,boolean,int,boolean) is gone Put.setTimeStamp() is gone org.apache.hadoop.hbase.filter.Filter has added getNextKeyHint(org.apache.hadoop.hbase.KeyValue) If you know the alternative to some of the old classes, please share. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932257#action_12932257 ] Hairong Kuang commented on HBASE-3234: -- I found out the problem. Looks that HDFS-724 missed a piece of code. PipelineAck#readFields does not serialize the Heartbeat ack correctly. I just checked our internal branch and I did correctly there. That's why our internal testing did not show this error. hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932259#action_12932259 ] stack commented on HBASE-3234: -- @Hairong Excellent. If you post a patch over in hdfs-724, I can try it for you in hbase. What about the bump in the transfer protocol version J-D identifies above? Could the hdfs-724 for 0.20-append branch not bumped the protocol? What you think? hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932261#action_12932261 ] Hairong Kuang commented on HBASE-3234: -- I attached a patch that fixes the problem: https://issues.apache.org/jira/secure/attachment/12459664/hbAckReply.patch. Could anybody give it a try? If it fixes the problem, I will commit it to append 0.20. hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3235) Intermittent incrementColumnValue failure in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932265#action_12932265
]
Gary Helmling commented on HBASE-3235:
--
Some further info on this. I added logging of the contents of
store.memstore.kvset when kvset.size() 1 in
testIncrementColumnValue_UpdatingInPlace(), like so:
{code}
long value = 1L;
long amount = 3L;
Put put = new Put(row);
put.add(fam1, qual1, Bytes.toBytes(value));
region.put(put);
long result = region.incrementColumnValue(row, fam1, qual1, amount, true);
assertEquals(value+amount, result);
Store store = region.getStore(fam1);
// ICV removes any extra values floating around in there.
if (store.memstore.kvset.size() 1) {
for (KeyValue kv : store.memstore.kvset) {
LOG.debug(memstore.kvset: row=+Bytes.toString(kv.getRow()) +
fam=+Bytes.toString(kv.getFamily())+
col=+Bytes.toString(kv.getQualifier())+
val=+Bytes.toLong(kv.getValue())+ ts=+kv.getTimestamp()+
memstore_ts=+kv.getMemstoreTS());
}
}
assertEquals(1, store.memstore.kvset.size());
assertTrue(store.memstore.snapshot.isEmpty());
{code}
With this in place, I get the following output for the failure case:
{noformat}
2010-11-15 15:36:24,242 DEBUG [main] regionserver.TestHRegion(1891):
memstore.kvset: row=rowA fam=colfamily1 col=qual1 val=1 ts=1289864184241
memstore_ts=1
2010-11-15 15:36:24,242 DEBUG [main] regionserver.TestHRegion(1891):
memstore.kvset: row=rowA fam=colfamily1 col=qual1 val=4 ts=1289864184241
memstore_ts=0
{noformat}
So it seems like it's only happening when the timestamps for the initial put
and the ICV are identical. In this case, the put gets memstoreTS=1 (from
RWCC.getWriteNumber()), but the ICV memstoreTS=0 in MemStore.upsert(List).
So, in MemStore.upsert(KeyValue), when we call kvset.tailSet(kv), we're missing
the initial put, because KeyValue.KV_COMPARATOR is evaluating it as less than
the ICV put, by inverting the memstoreTS comparison.
Given that that's the cause, what is the correct fix? Should the ICV put get
using RWCC.getWriteNumber() as well? Or should we be obtaining the tailset
without regards to the memstoreTS?
Intermittent incrementColumnValue failure in TestHRegion
Key: HBASE-3235
URL: https://issues.apache.org/jira/browse/HBASE-3235
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.0
Reporter: Gary Helmling
I first saw this in a Hudson build, but can reproduce locally with enough
test runs (5-10 times):
{noformat}
---
Test set: org.apache.hadoop.hbase.regionserver.TestHRegion
---
Tests run: 51, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 39.413 sec
FAILURE!
testIncrementColumnValue_UpdatingInPlace(org.apache.hadoop.hbase.regionserver.TestHRegion)
Time elapsed: 0.079 sec FAILURE!
junit.framework.AssertionFailedError: expected:1 but was:2
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:283)
at junit.framework.Assert.assertEquals(Assert.java:64)
at junit.framework.Assert.assertEquals(Assert.java:195)
at junit.framework.Assert.assertEquals(Assert.java:201)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testIncrementColumnValue_UpdatingInPlace(TestHRegion.java:1889)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
Alternately, the failure can also show up in
testIncrementColumnValue_UpdatingInPlace_Negative():
{noformat}
testIncrementColumnValue_UpdatingInPlace_Negative(org.apache.hadoop.hbase.regionserver.TestHRegion)
Time elapsed: 0.03 sec FAILURE!
junit.framework.AssertionFailedError: expected:2 but was:3
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:283)
at junit.framework.Assert.assertEquals(Assert.java:64)
at junit.framework.Assert.assertEquals(Assert.java:130)
at junit.framework.Assert.assertEquals(Assert.java:136)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.assertICV(TestHRegion.java:2081)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testIncrementColumnValue_UpdatingInPlace_Negative(TestHRegion.java:1990)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang reassigned HBASE-3234: Assignee: Hairong Kuang hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Assignee: Hairong Kuang Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932268#action_12932268 ] Hairong Kuang commented on HBASE-3234: -- For the version # bump problem, HDFS-724 is indeed an incompatible change. For append 0.20, it changes the semantics/syntax of heartbeat packets. hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Assignee: Hairong Kuang Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932273#action_12932273 ] stack commented on HBASE-3234: -- @Hairong OK. I don't think it'll be end of the world asking fellas to restart their clusters going between versions of the hadoop jar pre-724 and post-724. hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Assignee: Hairong Kuang Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2001) Coprocessors: Colocate user code with regions
[ https://issues.apache.org/jira/browse/HBASE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932285#action_12932285 ] HBase Review Board commented on HBASE-2001: --- Message from: st...@duboce.net bq. On 2010-10-05 23:10:58, stack wrote: bq. src/main/java/org/apache/hadoop/hbase/client/Action.java, line 30 bq. http://review.cloudera.org/r/876/diff/7/?file=14158#file14158line30 bq. bq. I took a look at the package-info.html. Very nice doc. One thought though was that the batch methods do not seem to be instrumented. Are they? The bulk of inserts are done by multiput now. bq. bq. Maybe link to the wiki page when you say this in package-info.html 'implement role-based access control for HBase' bq. bq. Fix this 'These code will be triggerd with existing...' bq. bq. BaseRegionObserver as the name of the class that implements BOTH Coprocessor and RegionObserver with sensible defaults seems off... it'd make sense as the name of an implemenation of RegionObserver but not of both. Is there a better name to give it -- even BaseRegionObserverCoprocessor? Unless BaseObserver already implements Coprocessor? bq. bq. Should this also say that methods can be new also? '...i.e., you can specify new passed parameters and return types for a method. ' bq. bq. CommandTarget is a strange name for an host of arbitrary user-designed methods. Can we come up w/ something more telling? Notions that come to mind are Substrate, Platform -- i.e. stuff you build up on. bq. bq. Minor.. fix '...the actually implemention class running...' bq. bq. Fix this '...How is the client side example of calling...' bq. bq. The example is missing a bit of code that would help along its illustration a few comments would help too but this is a minor criticism. Not important. I get the gist (Folks interested in CP need to start with this page -- it makes grokking the code the easier). bq. bq. This page would seem to indicate CPs can be chained. Am I reading that wrong? (See 'Load from configuration') Over in Gary review, he was saying on CP per region only. bq. bq. bq. Usually attribute names are upper-cased. Here we have 'Coprocessor$1' (that $1is intentional right?) bq. bq. This functionality, if its working, is amazing. bq. bq. bq. bq. bq. Mingjie Lai wrote: bq. @stack: bq. I didn't realize you posted a comment until last week, since your comments here didn't get pushed to jira, neither emails sent to d...@hbase. bq. bq. Thanks for your comments. I will address them very soon. But before that I'd like to finalize the name of ``CommandTarget'': bq. bq. You said, ``CommandTarget is a strange name for an host of arbitrary user-designed methods. Can we come up w/ something more telling? Notions that come to mind are Substrate, Platform -- i.e. stuff you build up on.'' bq. bq. Some of us suggested to use ``Endpoint'' instead of CommandTarget. Do you like it better? (I'm not really good at naming stuff) bq. bq. After finalizing the name, I will make the changes to both source code and package-info. And post a patch here. bq. bq. Thanks, bq. Mingjie bq. I'm not good at naming either Endpoint seems more 'generic', less loaded than 'CommandTarget'. If you fellas working with this stuff think that a better name then thats good by me. - stack --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/876/#review1438 --- Coprocessors: Colocate user code with regions - Key: HBASE-2001 URL: https://issues.apache.org/jira/browse/HBASE-2001 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Mingjie Lai Fix For: 0.92.0 Attachments: asm-transformations.pdf, HBASE-2001-RegionObserver-2.patch, HBASE-2001-RegionObserver.patch, HBASE-2001.patch.gz, packge-info.html, packge-info.html, packge-info.html Support user code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also. Use classloader which looks on HDFS. Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters). Not completely arbitrary code, should require implementation of an interface with callbacks for: * Open * Close * Split * Compact * (Multi)get and
[jira] Commented: (HBASE-2001) Coprocessors: Colocate user code with regions
[ https://issues.apache.org/jira/browse/HBASE-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932292#action_12932292 ] HBase Review Board commented on HBASE-2001: --- Message from: st...@duboce.net --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/876/#review1930 --- Ship it! +1 on commit to TRUNK. I think all below can be cleaned up on commit (Andrew, you going to commit?) src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java http://review.cloudera.org/r/876/#comment6139 Check in here. Looks like tabs? review board reporting it as whitespace. src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java http://review.cloudera.org/r/876/#comment6140 Usually in hbase code base there are spaces around operations; e.g. around '+'. src/main/java/org/apache/hadoop/hbase/client/coprocessor/ExecResult.java http://review.cloudera.org/r/876/#comment6142 Be careful. In hbase lines are 80 characters long normally. Fix on commit? src/main/java/org/apache/hadoop/hbase/client/coprocessor/ExecResult.java http://review.cloudera.org/r/876/#comment6143 I think its ok if these lines 80 characters src/main/java/org/apache/hadoop/hbase/client/coprocessor/package-info.java http://review.cloudera.org/r/876/#comment6144 Excellent src/main/java/org/apache/hadoop/hbase/coprocessor/package-info.java http://review.cloudera.org/r/876/#comment6146 Lots of white space in here. - stack Coprocessors: Colocate user code with regions - Key: HBASE-2001 URL: https://issues.apache.org/jira/browse/HBASE-2001 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Mingjie Lai Fix For: 0.92.0 Attachments: asm-transformations.pdf, HBASE-2001-RegionObserver-2.patch, HBASE-2001-RegionObserver.patch, HBASE-2001.patch.gz, packge-info.html, packge-info.html, packge-info.html Support user code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also. Use classloader which looks on HDFS. Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters). Not completely arbitrary code, should require implementation of an interface with callbacks for: * Open * Close * Split * Compact * (Multi)get and scanner next() * (Multi)put * (Multi)delete Add method to HTableInterface for invoking coprocessor methods and retrieving results. Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions for coprocessor methods and consistent/controlled access to internals: store access, threading, persistent and ephemeral state, scratch storage, etc. GitHub: https://github.com/trendmicro/hbase/tree/coprocessor Please see the latest attached package-info.html for updated description. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2002) Coprocessors: Client side support
[ https://issues.apache.org/jira/browse/HBASE-2002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932296#action_12932296 ] HBase Review Board commented on HBASE-2002: --- Message from: st...@duboce.net --- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/816/#review1933 --- Ship it! I did a quick pass over this. Most I'd seen already over in the Minjgie patch. I'm +1 getting it into TRUNK now early in the release cycle so probs. surface before release (You going to commit Andrew?). - stack Coprocessors: Client side support - Key: HBASE-2002 URL: https://issues.apache.org/jira/browse/HBASE-2002 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Gary Helmling Fix For: 0.92.0 High-level call interface for clients. Unlike RPC, calls addressed to rows or ranges of rows. Coprocessor client library resolves to actual locations. Calls across multiple rows automatically split into multiple parallelized RPCs Generic multicall RPC facility which incorporates this and multiget/multiput/multidelete and parallel scanners. Group and batch RPCs by region server. Track and retry outstanding RPCs. Ride over region relocations. Support addressing by explicit region identifier or by row key or row key range. Include a facility for merging results client side. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932300#action_12932300 ] stack commented on HBASE-3234: -- Tests are still running (its looking good). Not done yet. Will report back later. J-D and Todd, you Ok w/ version number going up when we have hdfs-724 in the hbase hacked hadoop jar again? hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Assignee: Hairong Kuang Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3234) hdfs-724 breaks TestHBaseTestingUtility multiClusters
[ https://issues.apache.org/jira/browse/HBASE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932315#action_12932315 ] Jean-Daniel Cryans commented on HBASE-3234: --- My only concern was that if we back out 724, then people won't be able to run on the tip of 0.20-append since the protocol won't be the same. hdfs-724 breaks TestHBaseTestingUtility multiClusters --- Key: HBASE-3234 URL: https://issues.apache.org/jira/browse/HBASE-3234 Project: HBase Issue Type: Bug Reporter: stack Assignee: Hairong Kuang Priority: Critical Fix For: 0.90.1 Attachments: org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility-output.txt, org.apache.hadoop.hbase.TestHBaseTestingUtility.txt We upgraded our hadoop jar in TRUNK to latest on 0.20-append branch. TestHBaseTestingUtility started failing reliably. If I back out hdfs-724, the test passes again. This issue is about figuring whats up here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3235) Intermittent incrementColumnValue failure in TestHRegion
[
https://issues.apache.org/jira/browse/HBASE-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12932357#action_12932357
]
HBase Review Board commented on HBASE-3235:
---
Message from: Gary Helmling ghelml...@gmail.com
---
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1224/
---
Review request for hbase and Ryan Rawson.
Summary
---
Fix for MemStore.upsert(KeyValue) to start the kvset.tailSet() of potential KVs
to remove at the beginning of entries for the row/family/qualifier combination,
ignoring timestamp to prevent Puts being skipped based on timestamp alone and
masking the ICV.
This addresses bug HBASE-3235.
http://issues.apache.org/jira/browse/HBASE-3235
Diffs
-
src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java b7409b0
src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java 7640997
Diff: http://review.cloudera.org/r/1224/diff
Testing
---
Added a new test:
TestHRegion.testIncrementColumnValue_UpdatingInPlace_TimestampClobber() to
recreate the existing failure condition: 1) put to a row/family/qualifier, 2)
ICV to the same row/family/qualifier with the same timestamp. This test fails
consistently without the patch to MemStore.
With the patch to MemStore, the new test case consistently passes. I also ran
TestHRegion 15+ times and saw no more intermittent failures of
testIncrementColumnValue_UpdatingInPlace(). Previously this was failing every
5 or so test runs, so this seems a pretty good indication it's fixed.
I also ran through the full test suite on 0.90 and all passed except for an
error in TestHLog...
Thanks,
Gary
Intermittent incrementColumnValue failure in TestHRegion
Key: HBASE-3235
URL: https://issues.apache.org/jira/browse/HBASE-3235
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.0
Reporter: Gary Helmling
I first saw this in a Hudson build, but can reproduce locally with enough
test runs (5-10 times):
{noformat}
---
Test set: org.apache.hadoop.hbase.regionserver.TestHRegion
---
Tests run: 51, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 39.413 sec
FAILURE!
testIncrementColumnValue_UpdatingInPlace(org.apache.hadoop.hbase.regionserver.TestHRegion)
Time elapsed: 0.079 sec FAILURE!
junit.framework.AssertionFailedError: expected:1 but was:2
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:283)
at junit.framework.Assert.assertEquals(Assert.java:64)
at junit.framework.Assert.assertEquals(Assert.java:195)
at junit.framework.Assert.assertEquals(Assert.java:201)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testIncrementColumnValue_UpdatingInPlace(TestHRegion.java:1889)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
Alternately, the failure can also show up in
testIncrementColumnValue_UpdatingInPlace_Negative():
{noformat}
testIncrementColumnValue_UpdatingInPlace_Negative(org.apache.hadoop.hbase.regionserver.TestHRegion)
Time elapsed: 0.03 sec FAILURE!
junit.framework.AssertionFailedError: expected:2 but was:3
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:283)
at junit.framework.Assert.assertEquals(Assert.java:64)
at junit.framework.Assert.assertEquals(Assert.java:130)
at junit.framework.Assert.assertEquals(Assert.java:136)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.assertICV(TestHRegion.java:2081)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testIncrementColumnValue_UpdatingInPlace_Negative(TestHRegion.java:1990)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (HBASE-3237) Split request accepted -- BUT CURRENTLY A NOOP
Split request accepted -- BUT CURRENTLY A NOOP -- Key: HBASE-3237 URL: https://issues.apache.org/jira/browse/HBASE-3237 Project: HBase Issue Type: Bug Affects Versions: 0.90.0 Reporter: Todd Lipcon Priority: Blocker Fix For: 0.90.0 The split button from the web UI displays this message and indeed seems to do nothing. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
