I've done a small proof of concept on some flume sink customizations in 1.3.1. I want to incorporate these customizations into our build server (Bamboo). I started by creating a local git repo of 1.4 and I'm able to run a mvn clean install just fine. However when I push the raw un-altered code to our build server the build fails and not consistently.
Half the time TestExecSource fails in flume-ng-core: build 01-Aug-2013 22:56:11 testMonitoredCounterGroup(org.apache.flume.source.TestExecSource) Time elapsed: 1264 sec <<< FAILURE! build 01-Aug-2013 22:56:11 org.junit.ComparisonFailure: Expected Value: 5 expected:<[5]> but was:<[0]> build 01-Aug-2013 22:56:11 at org.junit.Assert.assertEquals(Assert.java:125) build 01-Aug-2013 22:56:11 at org.apache.flume.source.TestExecSource.testMonitoredCounterGroup(TestExecSource.java:215) When that passes my build fails in flume-hdfs-sink build 02-Aug-2013 00:03:30 simpleHDFSGZipCompressedTest(org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster) Time elapsed: 2676 sec <<< ERROR! build 02-Aug-2013 00:03:30 java.lang.NullPointerException build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124) build 02-Aug-2013 00:03:30 at org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster.simpleHDFSGZipCompressedTest(TestHDFSEventSinkOnMiniCluster.java:164) ... build 02-Aug-2013 00:03:30 underReplicationTest(org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster) Time elapsed: 31 sec <<< ERROR! build 02-Aug-2013 00:03:30 java.io.IOException: Cannot lock storage target/test/dfs/dfs/name1. The directory is already locked. build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.lock(Storage.java:602) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1219) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1237) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1164) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:184) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:267) build 02-Aug-2013 00:03:30 at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124) I'm at a loss to explain the inconsistency of flume-ng-core and when it does succeed how to solve the failure of HDFS-sink. Any advice would be appreciated. Thanks, Mike This email and any files included with it may contain privileged, proprietary and/or confidential information that is for the sole use of the intended recipient(s). Any disclosure, copying, distribution, posting, or use of the information contained in or attached to this email is prohibited unless permitted by the sender. If you have received this email in error, please immediately notify the sender via return email, telephone, or fax and destroy this original transmission and its included files without reading or saving it in any manner. Thank you.
