Hello,
As promised, I come back with debugging details.
So:
******* In start-cluster.sh , the following echo's
echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
echo $HOSTLIST
echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
echo $FLINK_BIN_DIR
echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
# cluster mode, bring up job manager locally and a task manager on every slave
host
"$FLINK_BIN_DIR"/jobmanager.sh start cluster batch
==> gave:
start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/conf/slaves
start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/bin
start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
******* In jobmanager.sh , the following echo's
echo "Starting Job Manager"
echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
pwd
echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
echo ${FLINK_ENV_JAVA_OPTS}
echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
echo $FLINK_JM_CLASSPATH
echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
echo $INTERNAL_HADOOP_CLASSPATHS
echo "jobmanager ~~~~~~~~~~~~~~~~~~~~"
echo $FLINK_CONF_DIR
echo "~~~~~~~~~~~~~~~~~~~~"
$JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}"
-classpath "`manglePathList "$FLINK_JM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
org.apache.flink.runtime.jobmanager.JobManager --configDir "$FLINK_CONF_DIR"
--executionMode $EXECUTIONMODE --streamingMode "$STREAMINGMODE" > "$out" 2>&1 <
/dev/null &
==> gave:
jobmanager ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster
jobmanager ~~~~~~~~~~~~~~~~~~~~~
jobmanager ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar
jobmanager ~~~~~~~~~~~~~~~~~~~~~
::
jobmanager ~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/conf
~~~~~~~~~~~~~~~~~~~~
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/flink/runtime/jobmanager/JobManager
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.runtime.jobmanager.JobManager
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.jobmanager.JobManager.
Program will exit.
******* In taskmanager.sh, the following echo's
echo Starting task manager on host $HOSTNAME
echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
pwd
echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
echo ${FLINK_ENV_JAVA_OPTS}
echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
echo $FLINK_JM_CLASSPATH
echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
echo $INTERNAL_HADOOP_CLASSPATHS
echo "taskmanager ~~~~~~~~~~~~~~~~~~~~"
echo $FLINK_CONF_DIR
echo "~~~~~~~~~~~~~~~~~~~~"
$JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}"
-classpath "`manglePathList "$FLINK_TM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
org.apache.flink.runtime.taskmanager.TaskManager --configDir "$FLINK_CONF_DIR"
--streamingMode "$STREAMINGMODE" > "$out" 2>&1 < /dev/null &
==> gave:
taskmanager ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster
taskmanager ~~~~~~~~~~~~~~~~~~~~~
taskmanager ~~~~~~~~~~~~~~~~~~~~~
taskmanager ~~~~~~~~~~~~~~~~~~~~~
::
taskmanager ~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/conf
~~~~~~~~~~~~~~~~~~~~
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/flink/runtime/taskmanager/TaskManager
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.runtime.taskmanager.TaskManager
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class:
org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.
******* In flink , the following echo's
echo "flink ~~~~~~~~~~~~~~~~~~~~~"
pwd
echo "flink ~~~~~~~~~~~~~~~~~~~~~"
echo $CC_CLASSPATH
echo "flink ~~~~~~~~~~~~~~~~~~~~~"
echo $INTERNAL_HADOOP_CLASSPATHS
echo "~~~~~~~~~~~~~~~~~~~~"
$JAVA_RUN $JVM_ARGS "${log_setting[@]}" -classpath "`manglePathList
"$CC_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
org.apache.flink.client.CliFrontend "$@"
==> gave:
flink ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster
flink ~~~~~~~~~~~~~~~~~~~~~
/users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar
flink ~~~~~~~~~~~~~~~~~~~~~
::
~~~~~~~~~~~~~~~~~~~~
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/flink/client/CliFrontend
Caused by: java.lang.ClassNotFoundException: org.apache.flink.client.CliFrontend
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.client.CliFrontend. Program
will exit.
I hope that now you have a better image of the issue and I hope you can provide
some guidance to solve it.
I mention that I use the simplest scenario possible, just 2 nodes (1 master and
1 slave) without Hadoop for this test.
Also, I have a hostname command inside the script that shows me that I'm on the
master node when running these scripts.
Thank you very much!
Camelia
________________________________
From: Camelia Elena Ciolac
Sent: Wednesday, November 11, 2015 9:01 AM
To: [email protected]
Subject: RE: Cluster installation gives java.lang.NoClassDefFoundError for
everything
Good morning,
Thank you Stephan!
I keep on testing and in the meantime I'm wondering if the Java version on the
cluster may be part of the issue:
java version "1.6.0_36"
OpenJDK Runtime Environment (IcedTea6 1.13.8) (rhel-1.13.8.1.el6_7-x86_64)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
?
The cluster has shared network file system, so no problem in this aspect.
I keep on trying and I've put some echo's for debugging in the Flink start
scripts.
Best regards,
Camelia
________________________________
From: Camelia Elena Ciolac
Sent: Monday, November 09, 2015 2:32 PM
To: [email protected]
Subject: Cluster installation gives java.lang.NoClassDefFoundError for
everything
Hello,
I am configuring Flink to run on a cluster with NFS.
I have the Flink 0.9.1 distribution in some path in NFS and I added that path
in ~/.bashrc as FLINK_HOME, and also included the $FLINK_HOME/lib folder to
$PATH.
I have the slaves file and the yaml file configured correctly with the nodes
involved.
Still, when I run the following command I get the following errors:
$FLINK_HOME/bin/start-cluster.sh
(tried also
cd $FLINK_HOME
./bin/start-cluster.sh)
--> in /log/flink-theuser-jobmanager-thenode.out
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/flink/runtime/jobmanager/JobManager
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.runtime.jobmanager.JobManager
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.runtime.jobmanager.JobManager.
Program will exit.
-->in /log/flink-theuser-taskmanager-theslavenode.out
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/flink/runtime/taskmanager/TaskManager
Caused by: java.lang.ClassNotFoundException:
org.apache.flink.runtime.taskmanager.TaskManager
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class:
org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.
And lastly, in my script I have the run of a job which gives:
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/flink/client/CliFrontend
Caused by: java.lang.ClassNotFoundException: org.apache.flink.client.CliFrontend
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.flink.client.CliFrontend. Program
will exit.
What could be the cause and which is the solution?
Looking forward to your answer as soon as possible.
Many thanks,
Camelia
