Hi Metron Team,
I’m Dhruv, I’d love to contribute to Metron’s development. I’m a Solutions
Engineer at Hortonworks and I’m comfortable with Java, Storm, Maven, etc.
To kick things off I tried building and running Metron’s tests from
metron-streaming on my mac but I got 2 test failures. I looked at test reports
but couldn’t make out why this is happening. Is anyone else running into this
issue? My system is: Oracle JDK 1.8.0_60, Maven 3.3.3, OS X 10.11.3
Failed tests:
test(org.apache.metron.dataloads.hbase.mr.BulkLoadMapperIntegrationTest)
test(org.apache.metron.dataloads.hbase.mr.LeastRecentlyUsedPrunerIntegrationTest)
Tests run: 3, Failures: 2, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Metron-Streaming ................................... SUCCESS [ 0.744 s]
[INFO] Metron-Common ...................................... SUCCESS [ 8.255 s]
[INFO] Metron-EnrichmentAdapters .......................... SUCCESS [ 0.763 s]
[INFO] Metron-MessageParsers .............................. SUCCESS [ 1.479 s]
[INFO] Metron-Indexing .................................... SUCCESS [ 0.540 s]
[INFO] Metron-Alerts ...................................... SUCCESS [ 0.792 s]
[INFO] Metron-Testing ..................................... SUCCESS [ 0.414 s]
[INFO] Metron-DataLoads ................................... FAILURE [04:50 min]
[INFO] Metron-Topologies .................................. SKIPPED
[INFO] Metron-Pcap_Service ................................ SKIPPED
For BulkLoadMapperIntegrationTest:
$ cat
/Users/dkumar/code/incubator-metron-Metron_0.1BETA_rc7/metron-streaming/Metron-DataLoads/target/surefire-reports/org.apache.metron.dataloads.hbase.mr.BulkLoadMapperIntegrationTest.txt
-------------------------------------------------------------------------------
Test set: org.apache.metron.dataloads.hbase.mr.BulkLoadMapperIntegrationTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 105.608 sec <<<
FAILURE!
test(org.apache.metron.dataloads.hbase.mr.BulkLoadMapperIntegrationTest) Time
elapsed: 105.545 sec <<< FAILURE!
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.metron.dataloads.hbase.mr.BulkLoadMapperIntegrationTest.test(BulkLoadMapperIntegrationTest.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
Looks like some Hadoop job failed during the BulkLoadMapperIntegrationTest's
run:
2016-04-16 09:40:00,953 WARN [ContainersLauncher #0]
nodemanager.DefaultContainerExecutor
(DefaultContainerExecutor.java:launchContainer(213)) - Exception from
container-launch with container ID: container_1460824746646_0001_02_000001 and
exit code: 1
ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
In the LeastRecentlyUsedPrunerIntegrationTest’s report, I see these type of
messages indicating flaky Zookeeper connections:
2016-04-16 09:42:24,399 WARN
[localhost:50688.activeMasterManager-SendThread(localhost:50719)]
zookeeper.ClientCnxn (ClientCnxn.java:run(1102)) - Session 0x1541ff06a570005
for server null, unexpected error, closing socket connection and attempting
reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
Any ideas? I can rerun and send more logs if needed.
Thanks,
Dhruv
