[jira] [Commented] (HBASE-5751) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257096#comment-13257096 ] Gregory Chanan commented on HBASE-5751: --- I don't think HBASE-5213 caused TestLogRolling to fail on 0.90. Here's my evidence: 1) Jon reverted it on April 6th, but TestLogRolling has hung since then: https://builds.apache.org/view/G-L/view/HBase/job/hbase-0.90/471/ https://builds.apache.org/view/G-L/view/HBase/job/hbase-0.90/472/ 2) It runs fine for me locally with the patch applied 3) TestLogRolling doesn't use backup masters, which is the part of the code that HBASE-5213 touches. hbase master stop does not bring down backup masters -- Key: HBASE-5751 URL: https://issues.apache.org/jira/browse/HBASE-5751 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Gregory Chanan Fix For: 0.90.7 Carry forward the discussion from parent for 0.90 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5751) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257224#comment-13257224 ] Gregory Chanan commented on HBASE-5751: --- It looks like it failed 4 times in a row after the commit, and it has currently failed 3 times in a row [see 471-473]. The current failure string is long after the revert (see below). Perhaps we should open an issue to figure out the failure? I'm fine with HBASE-5213 not going into 0.90 until after that is figured out. commit b05142a79fb3b461081d20c4da3592ed59a956b9 Author: Jonathan Hsieh jmhs...@apache.org Date: Fri Apr 6 16:35:26 2012 + Revert 'HBASE-5213 hbase master stop does not bring down backup masters' due to hanging TestLogRolling git-svn-id: https://svn.apache.org/repos/asf/hbase/branches/0.90@1310456 13f79535-47bb-0310-9956-ffa450edef68 hbase master stop does not bring down backup masters -- Key: HBASE-5751 URL: https://issues.apache.org/jira/browse/HBASE-5751 Project: HBase Issue Type: Sub-task Reporter: Lars Hofhansl Assignee: Gregory Chanan Fix For: 0.90.7 Carry forward the discussion from parent for 0.90 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4254) Get tests passing on Hadoop 23
[ https://issues.apache.org/jira/browse/HBASE-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242122#comment-13242122 ] Gregory Chanan commented on HBASE-4254: --- I did some playing around with TestLogRolling.testLogRollOnPipelineRestart against 0.23. It looks like calling recoverLease on the HLog path and waiting awhile before trying to create the reader works. If you don't call recoverLease the reader will never succeed. Let's assume, for the sake of argument, that replicas waiting recovery will not return a full visible length. 1) Are we supposed to have to call recoverLease? (I'm guessing the recovery design doc covers this, I'll check it out) 2) Where should we call recoverLease? Just in the test? In HLog.createWriter? (it would block for awhile if we actually waited for the recovery)? Get tests passing on Hadoop 23 -- Key: HBASE-4254 URL: https://issues.apache.org/jira/browse/HBASE-4254 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Fix For: 0.92.2 Currently some 30 or so tests are failing on the HBase-trunk-on-hadoop-23 build. It looks like most are reflection-based issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5608) MR testcases are failing in QA builds
[ https://issues.apache.org/jira/browse/HBASE-5608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13234831#comment-13234831 ] Gregory Chanan commented on HBASE-5608: --- I looked at these briefly; you are correct in that the MR job fails (basically) silently. I did some work on these test cases against hadoop-0.23, see https://issues.apache.org/jira/browse/HBASE-5317 Against 0.23 I needed to look in the target/org.apache.hadoop.mapred.MiniMRCluster directory for the MR logs. The problem is: 1) I'm not sure this directory is generated against hadoop-1.0.0 (the default jenkins build), so the MR logs may be somewhere else 2) We need someone with access to the jenkins machine to search around and pull the logs If someone with access to the jenkins machine can look around and find the logs, I'd be happy to take it from there. MR testcases are failing in QA builds - Key: HBASE-5608 URL: https://issues.apache.org/jira/browse/HBASE-5608 Project: HBase Issue Type: Bug Components: build, mapreduce, test Affects Versions: 0.92.2 Environment: Hadoop QA - precommit builds Reporter: Laxman Priority: Blocker Labels: build-failure, mapreduce, test-fail Many of the MR testcases are failing in PreCommit builds (triggered by Hadoop QA). Failing testcases are a) TestImportTsv b) TestHFileOutputFormat c) TestTableMapReduce -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5614) createTableAsync gets blocked and it is not an asynchronous operation
[
https://issues.apache.org/jira/browse/HBASE-5614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13235199#comment-13235199
]
Gregory Chanan commented on HBASE-5614:
---
Are you going to look into fixing this Mubarak? If not, I'll take a look.
createTableAsync gets blocked and it is not an asynchronous operation
-
Key: HBASE-5614
URL: https://issues.apache.org/jira/browse/HBASE-5614
Project: HBase
Issue Type: Bug
Components: master, regionserver
Affects Versions: 0.90.4
Reporter: Mubarak Seyed
Labels: noob
I have created table A using {{HBaseAdmin.createTableAsync()}} with 2000
pre-split regions, the expectation was client RPC call does not get blocked
and i had only used 5 minutes as RPC socket timeout value.
It appears from logs/code that clients gets blocked (honoring the
{{hbase.rpc.timeout}} value), times out after 5 minutes, master nio channel
gets closed (related to client which invoked the createAsyncTable operation),
put region entries in inconsistent state in META, meaning {{info:server}} is
marked as null and AssignmentManager gets null server value (which causes
NPE) and causes to table creation fail, i could not succeed until changing
the {{hbase.rpc.timeout}} value to 15 minutes and created tables one at a
time (each table has more than 2K pre-split regions in it)
Please refer the logs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226353#comment-13226353 ] Gregory Chanan commented on HBASE-5213: --- Any thoughts regarding 92 or 90? Thanks for the reviews/commit. hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5529) MR test failures becuase MALLOC_ARENA_MAX is not set
[ https://issues.apache.org/jira/browse/HBASE-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225023#comment-13225023 ] Gregory Chanan commented on HBASE-5529: --- I don't think we need to add it to hbase-env.sh. The issue (at least what I've experienced) is with the mapreduce containers. So we need it in the surefire config when running unit tests, but when running the jobs normally, hadoop already has the configuration, because of HADOOP-7154. If we run into these errors in another area of hbase, we can add it at that time. MR test failures becuase MALLOC_ARENA_MAX is not set Key: HBASE-5529 URL: https://issues.apache.org/jira/browse/HBASE-5529 Project: HBase Issue Type: Bug Components: mapreduce, test Affects Versions: 0.92.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Attachments: HBASE-5529-to92.patch, HBASE-5529-trunk.patch When running unit tests on CentOS 6 I get a bunch of unit test failures in mapreduce-related tests due to: 2012-03-03 00:14:18,776 WARN [Container Monitor] monitor.ContainersMonitorImpl$MonitoringThread(436): Container [pid=21446,containerID=container_1330762435849_0002_01_01] is running beyond virtual memory limits. Current usage: 223.1mb of 2.0gb physical memory used; 6.9gb of 4.2gb virtual memory used. Killing container. Note: this also came up in the mapreduce project. See: https://issues.apache.org/jira/browse/MAPREDUCE-3933 Patch coming shortly -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5529) MR test failures becuase MALLOC_ARENA_MAX is not set
[ https://issues.apache.org/jira/browse/HBASE-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13223534#comment-13223534 ] Gregory Chanan commented on HBASE-5529: --- The 0.94 patch would be identical to trunk. Do you want me to reattach with a 0.94 name? Regarding your questions, see here: https://issues.apache.org/jira/browse/HADOOP-7154 That is a better explanation than I gave or the linked MR jira gives. I think that addresses all your questions, but let me know if you have more. MR test failures becuase MALLOC_ARENA_MAX is not set Key: HBASE-5529 URL: https://issues.apache.org/jira/browse/HBASE-5529 Project: HBase Issue Type: Bug Components: mapreduce, test Affects Versions: 0.92.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Attachments: HBASE-5529-to92.patch, HBASE-5529-trunk.patch When running unit tests on CentOS 6 I get a bunch of unit test failures in mapreduce-related tests due to: 2012-03-03 00:14:18,776 WARN [Container Monitor] monitor.ContainersMonitorImpl$MonitoringThread(436): Container [pid=21446,containerID=container_1330762435849_0002_01_01] is running beyond virtual memory limits. Current usage: 223.1mb of 2.0gb physical memory used; 6.9gb of 4.2gb virtual memory used. Killing container. Note: this also came up in the mapreduce project. See: https://issues.apache.org/jira/browse/MAPREDUCE-3933 Patch coming shortly -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214998#comment-13214998 ] Gregory Chanan commented on HBASE-5317: --- Okay, this looks like a mvn issue. On linux, if JAVA_HOME is not set, mvn sets JAVA_HOME to be whatever shows up in mvn --version On mac, mvn appears to not set JAVA_HOME if it is not already set. So if you set JAVA_HOME to /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home the test should pass. The new version of mapreduce requires JAVA_HOME to be set and tries to execute $JAVA_HOME + bin/java -- they should probably report a bug if JAVA_HOME is not set. Let me know if you want me to do anything else wrt this issue. Fix TestHFileOutputFormat to work against hadoop 0.23 - Key: HBASE-5317 URL: https://issues.apache.org/jira/browse/HBASE-5317 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0, 0.94.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92: Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable It looks like on trunk, this also results in an error: testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13215066#comment-13215066 ] Gregory Chanan commented on HBASE-5317: --- I am unable to reproduce either on my machine or on a coworker's Mac. Is it possible for you to try it on another machine? Perhaps some (security?) setting on your machine makes mapreduce unable to open the required port or something? Fix TestHFileOutputFormat to work against hadoop 0.23 - Key: HBASE-5317 URL: https://issues.apache.org/jira/browse/HBASE-5317 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0, 0.94.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92: Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable It looks like on trunk, this also results in an error: testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214025#comment-13214025 ] Gregory Chanan commented on HBASE-5317: --- @Ted: Checked it on a mac, seeing the same failure as you. Can you confirm something for me? If you cd to target/org.apache.hadoop.mapred.MiniMRCluster and run the following commands, do you see something similar? $ find . -name stderr ./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_2/application_1329939539132_0001/container_1329939539132_0001_01_01/stderr find . -name syslog $ $ cat ./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_2/application_1329939539132_0001/container_1329939539132_0001_01_01/stderr /bin/bash: /bin/java: No such file or directory Fix TestHFileOutputFormat to work against hadoop 0.23 - Key: HBASE-5317 URL: https://issues.apache.org/jira/browse/HBASE-5317 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0, 0.94.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92: Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable It looks like on trunk, this also results in an error: testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13214353#comment-13214353 ] Gregory Chanan commented on HBASE-5317: --- Thanks very much, Ted. Could also you post the output of mvn --version where you ran the test? I'm particularly interested in the value of Java home: -- is there something valid at the location it is giving you? Fix TestHFileOutputFormat to work against hadoop 0.23 - Key: HBASE-5317 URL: https://issues.apache.org/jira/browse/HBASE-5317 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.0, 0.94.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92: Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable It looks like on trunk, this also results in an error: testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[
https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13213212#comment-13213212
]
Gregory Chanan commented on HBASE-5317:
---
@Ted:
Hmm...just to check I fetched the latest applied the patch and both:
{code}
mvn -Dhadoop.profile=23 clean test -P localTests
-Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
{code}
and
{code}
mvn clean test -P localTests
-Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
{code}
passed. Can you clarify exactly what you ran?
I don't know what is going on. I've noticed those tests have been failing
Hadoop QA as well for a few days, and I can't reproduce those either. Do you
have any idea what is going on?
Fix TestHFileOutputFormat to work against hadoop 0.23
-
Key: HBASE-5317
URL: https://issues.apache.org/jira/browse/HBASE-5317
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.92.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch,
HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch,
HBASE-5317-v6.patch
Running
mvn -Dhadoop.profile=23 test -P localTests
-Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
yields this on 0.92:
Failed tests:
testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
HFile for column family info-A not found
Tests in error:
test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
/home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0
(Is a directory)
testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
It looks like on trunk, this also results in an error:
testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but
haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5346) Fix testColumnFamilyCompression and test_TIMERANGE in TestHFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-5346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209952#comment-13209952 ] Gregory Chanan commented on HBASE-5346: --- Thanks! Can we get this into 0.92 as well? Do I have to do anything to make that happen? Fix testColumnFamilyCompression and test_TIMERANGE in TestHFileOutputFormat Key: HBASE-5346 URL: https://issues.apache.org/jira/browse/HBASE-5346 Project: HBase Issue Type: Sub-task Components: mapreduce, test Affects Versions: 0.94.0, 0.92.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Fix For: 0.94.0 Attachments: HBASE-5346-v0.patch Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92 (for testColumnFamilyCompression and test_TIMERANGE): Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) The problem is that these tests make incorrect assumptions about the output of mapreduce jobs. Prior to 0.23, temporary data was in, for example: ./_temporary/_attempt___r_00_0/b/1979617994050536795 Now that has changed. The correct way to get that path is based on getDefaultWorkFile. Also, the data is not moved into the outputPath until both the Task and Job are committed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5346) Fix testColumnFamilyCompression and test_TIMERANGE in TestHFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-5346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13209019#comment-13209019 ] Gregory Chanan commented on HBASE-5346: --- @stack: Any comment on my previous message? It looks like this can be committed, but my reading of the results could be wrong. Fix testColumnFamilyCompression and test_TIMERANGE in TestHFileOutputFormat Key: HBASE-5346 URL: https://issues.apache.org/jira/browse/HBASE-5346 Project: HBase Issue Type: Sub-task Components: mapreduce, test Affects Versions: 0.90.4, 0.92.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Attachments: HBASE-5346-v0.patch Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92 (for testColumnFamilyCompression and test_TIMERANGE): Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) The problem is that these tests make incorrect assumptions about the output of mapreduce jobs. Prior to 0.23, temporary data was in, for example: ./_temporary/_attempt___r_00_0/b/1979617994050536795 Now that has changed. The correct way to get that path is based on getDefaultWorkFile. Also, the data is not moved into the outputPath until both the Task and Job are committed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207090#comment-13207090 ] Gregory Chanan commented on HBASE-5317: --- @Evert: Yes, that is the idea. I'm hoping to have a patch that does that and solves the jar issue above by the end of today. Fix TestHFileOutputFormat to work against hadoop 0.23 - Key: HBASE-5317 URL: https://issues.apache.org/jira/browse/HBASE-5317 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0, 0.92.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92: Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable It looks like on trunk, this also results in an error: testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[
https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13207272#comment-13207272
]
Gregory Chanan commented on HBASE-5317:
---
@Ted:
I tried approach #1, but it seems like it is not kosher to have the test depend
on the maven packaging step. For example:
{code}
mvn -Dhadoop.profile=23 -DskipTests package
mvn -Dhadoop.profile=23 test ...
{code}
works, but
{code}
mvn -Dhadoop.profile=23 clean test ...
{code}
fails because the jars aren't even built without the package step.
#2 will probably work in both cases. It don't think it's great either, because
ideally the tests should exercise the jars, because that is what downstream
users would be running against. I think testing against the jars requires
making hbase a multi-module project and is therefore probably:
0) outside the scope of this JIRA
1) a lot of work
2) should be discussed in the community
Therefore, I propose getting attempt #2 to work and filing a JIRA to figure out
a better packaging story so we can later get rid of #2.
Fix TestHFileOutputFormat to work against hadoop 0.23
-
Key: HBASE-5317
URL: https://issues.apache.org/jira/browse/HBASE-5317
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.92.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
Running
mvn -Dhadoop.profile=23 test -P localTests
-Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
yields this on 0.92:
Failed tests:
testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
HFile for column family info-A not found
Tests in error:
test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
/home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0
(Is a directory)
testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
It looks like on trunk, this also results in an error:
testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but
haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[
https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13205982#comment-13205982
]
Gregory Chanan commented on HBASE-5317:
---
Ok, did some more digging.
In 0.23 the MRAppMaster tries to load up the OutputFileFormat, so we have to
load that, which we didn't have to do prior to 0.23.
It looks like: TableMapReduceUtil.addDependencyJars(job)
is supposed to do this, but when I run it, no jars can be found for the classes
associated with the job.
By default, it looks like the classpath in the test is:
{basedir}/target/test-classes:{basedir}/target/classes:...
but the relevant jars are in:
target/
target/hbase-{VERSION}/hbase-{VERSION}/
By the way, when I add hbase jars manually to tmpjars [which is what
addDependencyJars does], the test passes.
So it seems like the solution is either:
1) Modify the classpath to include those jars
2) Build a jar myself of the relevant classes
I'm not sure what the correct answer is; please advise.
Fix TestHFileOutputFormat to work against hadoop 0.23
-
Key: HBASE-5317
URL: https://issues.apache.org/jira/browse/HBASE-5317
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.92.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
Running
mvn -Dhadoop.profile=23 test -P localTests
-Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
yields this on 0.92:
Failed tests:
testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
HFile for column family info-A not found
Tests in error:
test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
/home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0
(Is a directory)
testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
It looks like on trunk, this also results in an error:
testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but
haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5346) Fix testColumnFamilyCompression and test_TIMERANGE in TestHFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-5346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204775#comment-13204775 ] Gregory Chanan commented on HBASE-5346: --- It looks like the tests testColumnFamilyCompression and test_TIMERANGE passed, if my reading of the results is correct. Fix testColumnFamilyCompression and test_TIMERANGE in TestHFileOutputFormat Key: HBASE-5346 URL: https://issues.apache.org/jira/browse/HBASE-5346 Project: HBase Issue Type: Sub-task Components: mapreduce, test Affects Versions: 0.90.4, 0.92.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Attachments: HBASE-5346-v0.patch Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92 (for testColumnFamilyCompression and test_TIMERANGE): Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) The problem is that these tests make incorrect assumptions about the output of mapreduce jobs. Prior to 0.23, temporary data was in, for example: ./_temporary/_attempt___r_00_0/b/1979617994050536795 Now that has changed. The correct way to get that path is based on getDefaultWorkFile. Also, the data is not moved into the outputPath until both the Task and Job are committed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[
https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13202813#comment-13202813
]
Gregory Chanan commented on HBASE-5317:
---
The new MiniMRCluster in 0.23 copies some new MrApp jars into the working
directory of the filesystem (which is the home directory of that file system):
{code}
FileSystem fs = FileSystem.get(conf);
Path testRootDir = new Path(target, caller.getName() + -tmpDir)
.makeQualified(fs);
Path appJar = new Path(testRootDir, MRAppJar.jar);
// Copy MRAppJar and make it private.
Path appMasterJar = new Path(MiniMRYarnCluster.APPJAR);
fs.copyFromLocalFile(appMasterJar, appJar);
{code}
This also happens to be the HBASE_DIR:
{code}
Path hbaseRootdir = fs.makeQualified(fs.getHomeDirectory());
this.conf.set(HConstants.HBASE_DIR, hbaseRootdir.toString());
{code}
I created a subdirectory under fs.getHomeDirectory called hbase and made this
the HBASE_DIR. This gets past the TestTable issue and back to the
ClassNotFoundException.
Since these issues appear to be issues with the test utilities, rather than the
test themselves, I am going to create a subtask for fixing the test themselves
(which will just be the v0 patch).
Fix TestHFileOutputFormat to work against hadoop 0.23
-
Key: HBASE-5317
URL: https://issues.apache.org/jira/browse/HBASE-5317
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.92.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
Running
mvn -Dhadoop.profile=23 test -P localTests
-Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
yields this on 0.92:
Failed tests:
testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
HFile for column family info-A not found
Tests in error:
test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
/home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0
(Is a directory)
testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
It looks like on trunk, this also results in an error:
testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but
haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201473#comment-13201473 ] Gregory Chanan commented on HBASE-5317: --- @Ted: I agree we shouldn't just catch the TableExistsException -- that's why I didn't initially post that part of the patch. My recollection of that issue was that the MiniMRCluster is creating a target directory in hbase.rootdir [I could be wrong about the exact location]. When we call table.getTableDescriptor() it can't get a table descriptor for target, so throws a TableExistsException. Can the handleDeprecation call prevent the target directory from being created? Or are you thinking of something else? It also seems a little strange to me that calling table.getTableDescriptor(); tries to get table descriptors for *everything* in hbase.rootdir. Why should I get an exception thrown if hbase can't find a TableDescriptor for target when I am only asking about table? Fix TestHFileOutputFormat to work against hadoop 0.23 - Key: HBASE-5317 URL: https://issues.apache.org/jira/browse/HBASE-5317 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0, 0.92.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92: Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable It looks like on trunk, this also results in an error: testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[
https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201841#comment-13201841
]
Gregory Chanan commented on HBASE-5317:
---
@Ted:
FSUtils.getTableDirs() excludes a specific list of directories. Specifically:
{code}
Arrays.asList(new String[]{ HREGION_LOGDIR_NAME, HREGION_OLDLOGDIR_NAME,
CORRUPT_DIR_NAME, Bytes.toString(META_TABLE_NAME),
Bytes.toString(ROOT_TABLE_NAME), SPLIT_LOGDIR_NAME }));
{code}
So if the MiniMRCluster creates a target directory, it will be returned.
Fix TestHFileOutputFormat to work against hadoop 0.23
-
Key: HBASE-5317
URL: https://issues.apache.org/jira/browse/HBASE-5317
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.92.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
Running
mvn -Dhadoop.profile=23 test -P localTests
-Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
yields this on 0.92:
Failed tests:
testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
HFile for column family info-A not found
Tests in error:
test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
/home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0
(Is a directory)
testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
It looks like on trunk, this also results in an error:
testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat):
TestTable
I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but
haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200632#comment-13200632 ] Gregory Chanan commented on HBASE-5317: --- @Ted: your output is correct. There is another patch required to get past the TestTable error and to the Error starting MRAppMaster. I didn't include it because it doesn't solve the issue, only lets you further along. I wanted to leave the patch in a fresh state that actually solves the issue addressed by the JIRA. Perhaps the best thing to do is narrow the scope of this JIRA (to just get test_TIMERANGE and testColumnFamilyCompression to pass) and then open a new JIRA to solve the remaining issues. Do you think that makes the most sense? Fix TestHFileOutputFormat to work against hadoop 0.23 - Key: HBASE-5317 URL: https://issues.apache.org/jira/browse/HBASE-5317 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0, 0.92.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Attachments: HBASE-5317-v0.patch Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92: Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable It looks like on trunk, this also results in an error: testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200661#comment-13200661 ] Gregory Chanan commented on HBASE-5317: --- @Ted: Yes, that is the problem that causes the TestTable error. I'll post a patch for that tomorrow. Fix TestHFileOutputFormat to work against hadoop 0.23 - Key: HBASE-5317 URL: https://issues.apache.org/jira/browse/HBASE-5317 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0, 0.92.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Attachments: HBASE-5317-v0.patch Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92: Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable It looks like on trunk, this also results in an error: testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work against hadoop 0.23
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13200303#comment-13200303 ] Gregory Chanan commented on HBASE-5317: --- testMRIncrementalLoad (and probably the other 2) fail with TestTable; this is pretty easy to get past. But after that MR Job fails with: 2012-02-03 17:02:37,982 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster org.apache.hadoop.yarn.YarnException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.mapreduce.HFileOutputFormat at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:320) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.init(MRAppMaster.java:213) Fix TestHFileOutputFormat to work against hadoop 0.23 - Key: HBASE-5317 URL: https://issues.apache.org/jira/browse/HBASE-5317 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0, 0.92.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Attachments: HBASE-5317-v0.patch Running mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat yields this on 0.92: Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found Tests in error: test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_00_0 (Is a directory) testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable It looks like on trunk, this also results in an error: testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5212) Fix test TestTableMapReduce against 0.23.
[ https://issues.apache.org/jira/browse/HBASE-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13197612#comment-13197612 ] Gregory Chanan commented on HBASE-5212: --- @Zhihong: good point. It looks like testLogRollOnPipelineRestart is a separate issue, see https://issues.apache.org/jira/browse/HDFS-2288 https://issues.apache.org/jira/browse/HBASE-4254 Todd has brought that issue back up. I think this patch addresses: 1) HBASE-5212 since it does fix TestTableMapReduce against 0.23 2) HBASE-5191 since it fixes the compilation issue and causes that test (testLogRollOnDatanodeDeath) to pass against 0.23. Also, I ran the above failed unit tests locally with the patch applied (not against 0.23) and they passed. Fix test TestTableMapReduce against 0.23. - Key: HBASE-5212 URL: https://issues.apache.org/jira/browse/HBASE-5212 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Mahadev konar Assignee: Gregory Chanan Fix For: 0.92.1 Attachments: 5212-v2.txt, HBASE-5212-v3.patch, HBASE-5212.patch As reported by Andrew on the hadoop mailing list, mvn -Dhadoop.profile=23 clean test -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce fails on 0.92 branch. There are minor changes to HBase poms required to fix that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
