The exceptions you see are warnings and an issue for the same is still
open at HBASE-4709 <https://issues.apache.org/jira/browse/HBASE-4709> .
You would see a lot of exceptions but most of these are going to be
warnings.
The reason why it did not work earlier is due to error in verify
permissions.
3290 [main] WARN org.apache.hadoop.hdfs.server.datanode.DataNode - Invalid
directory in dfs.data.dir: Incorrect permission for
/tmp/incubator-crunch/crunch/target/test-data/65210a78-e0a6-4435-97fd-a7e27173fafb/dfs/data/data1,
expected: rwxr-xr-x, while actual: rwxrwxr-x
3299 [main] WARN org.apache.hadoop.hdfs.server.datanode.DataNode - Invalid
directory in dfs.data.dir: Incorrect permission for
/tmp/incubator-crunch/crunch/target/test-data/65210a78-e0a6-4435-97fd-a7e27173fafb/dfs/data/data2,
expected: rwxr-xr-x, while actual: rwxrwxr-x
3299 [main] ERROR org.apache.hadoop.hdfs.server.datanode.DataNode - All
directories in dfs.data.dir are invalid.
The same has been corrected in patchs of CRUNCH-4 by setting the correct
set of permissions.
conf.set("dfs.datanode.data.dir.perm", "775");
regards
Rahul
On 05-08-2012 03:00, Matthias Friedrich wrote:
Good idea, that did it. Still lots of exceptions but it's not the
time of day to worry about that.
Thanks,
Matthias
On Saturday, 2012-08-04, Josh Wills wrote:
Hrm. Probably requires the patch from CRUNCH-4?
On Sat, Aug 4, 2012 at 2:04 PM, Matthias Friedrich <[email protected]> wrote:
Nope, same thing with every single version starting from 0.90.0.
On Saturday, 2012-08-04, Josh Wills wrote:
Try 0.90.4, it worked for me, IIRC.
On Sat, Aug 4, 2012 at 1:50 PM, Matthias Friedrich <[email protected]> wrote:
Hi,
I tried getting the test suite running with Hadoop 1.0.3 and HBase
0.90.5 for CRUNCH-16. All tests passed except for the HBase word
count (log attached). I'm no HBase expert, but perhaps someone else
knows what's going wrong here.
To reproduce simply change version numbers (see attached patch) and
run the following command:
mvn -Dit.test=org.apache.crunch.WordCountHBaseIT verify
BTW, HBase is really trying my patience today, it causes a dozen
dependency conflicts with Hadoop, half of which have no clean
resolution.
Thanks in advance,
Matthias
