Hi guys, Update: Turns out there was a mistake in the way I was debugging. The path in the commands that are executed are relative and I wasn't launching them from the right location hence the class not found error.
When I launched it from the correct location, I got a Major.minor 51 error. The command that the supervisor is using to launch the workers is /usr/jdk64/.../bin/java which is a JDK 1.6. However, my JAVA_HOME points to JDK 1.7. I don't know why (and from where) it's taking the JDK 1.6 path. My update-alternatives -config java is also set to 1.7. I did a very dirty hack, replacing the JDK 1.6 folder contents with that of JDK 1.7 and my workers are launching just fine now. Clearly, I can't do this on all nodes. Any idea why it's taking the wrong path? The supervisor logs show that the Java version is set to the correct version. Log below: This: 2014-03-13 11:05:40 o.a.z.ZooKeeper [INFO] Client environment:java.version=1.7.0_51 and this: 2014-03-13 11:05:40 o.a.z.ZooKeeper [INFO] Client environment:java.home=/usr/java/jdk1.7.0_51/jre Any idea where it's picking up 1.6 from? Thanks in advance! -- Thanks & Regards, Gufran Pathan | +91-9566811502 | www.mu-sigma.com<http://www.mu-sigma.com/> | From: Gufran Pathan Sent: Thursday, March 13, 2014 11:09 PM To: [email protected] Subject: Workers not starting (backtype.storm.daemon.worker class not found) Hi, I'm a newbie with Storm and I have setup a Storm-on-Yarn on an HDP cluster using the instructions at http://hortonworks.com/kb/storm-on-yarn-install-on-hdp2-beta-cluster/ and the storm-yarn-master from https://github.com/anfeng/storm-yarn. I'm able to get Nimbus running and even submit topologies and see them on Storm UI. However, the spouts and the bolts don't seem to be "working" (0 counts of tuples emitted). I did some digging around and realized that my worker daemons are not starting. The supervisor log spits out these: 2014-03-13 11:22:03 b.s.d.supervisor [INFO] 18bf93a1-1cea-4e99-93da-8f36a4e9c056 still hasn't started I tried launching the worker command from the "Launching worker with command" line in the supverviser log and I got this error: Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/daemon/worker Caused by: java.lang.ClassNotFoundException: backtype.storm.daemon.worker at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) Could not find the main class: backtype.storm.daemon.worker. Program will exit. It looks like it can't find the worker class although it's present in the storm-core jar. Any ideas on how I can proceed with troubleshooting this? I've attached the nimbus and the supervisor logs. -- Thanks & Regards, Gufran Pathan | +91-9566811502 | www.mu-sigma.com<http://www.mu-sigma.com/> | Disclaimer: http://www.mu-sigma.com/disclaimer.html
