Hi Camelia,
Flink 0.9.X supports Java 6. So this can't be the issue.
Out of curiosity, I gave it a spin on a Linux machine with OpenJDK 6. I was
able to start the command-line interface, job manager and task managers.
java version "1.6.0_36"
OpenJDK Runtime Environment (IcedTea6 1.13.8) (6b36-1.13.8-0ubuntu1~14.04)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
I think the error must be caused by your NFS setup. Let's start with
./bin/flink. For instance, can you access
"/users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar" from the
machine where you run ./bin/flink?
Best,
Max
On Wed, Nov 11, 2015 at 10:41 AM, Camelia Elena Ciolac <came...@chalmers.se>
wrote:
> Hello,
>
> As promised, I come back with debugging details.
> So:
>
> ******* In start-cluster.sh , the following echo's
>
> echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
> echo $HOSTLIST
> echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
> echo $FLINK_BIN_DIR
> echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
>
> # cluster mode, bring up job manager locally and a task manager on every
> slave host
> "$FLINK_BIN_DIR"/jobmanager.sh start cluster batch
>
> ==> gave:
>
> start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /users/camelia/thecluster/flink-0.9.1/conf/slaves
> start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> /users/camelia/thecluster/flink-0.9.1/bin
> start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
> ******* In jobmanager.sh , the following echo's
>
> echo "Starting Job Manager"
> echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
> pwd
> echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
> echo ${FLINK_ENV_JAVA_OPTS}
> echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
> echo $FLINK_JM_CLASSPATH
> echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
> echo $INTERNAL_HADOOP_CLASSPATHS
> echo "jobmanager ~~~~~~~~~~~~~~~~~~~~"
> echo $FLINK_CONF_DIR
> echo "~~~~~~~~~~~~~~~~~~~~"
> $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}"
> -classpath "`manglePathList
> "$FLINK_JM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
> org.apache.flink.runtime.jobmanager.JobManager --configDir
> "$FLINK_CONF_DIR" --executionMode $EXECUTIONMODE --streamingMode
> "$STREAMINGMODE" > "$out" 2>&1 < /dev/null &
>
>
>
> ==> gave:
>
> jobmanager ~~~~~~~~~~~~~~~~~~~~~
> /users/camelia/thecluster
> jobmanager ~~~~~~~~~~~~~~~~~~~~~
>
> jobmanager ~~~~~~~~~~~~~~~~~~~~~
>
> /users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar
> jobmanager ~~~~~~~~~~~~~~~~~~~~~
> ::
> jobmanager ~~~~~~~~~~~~~~~~~~~~
> /users/camelia/thecluster/flink-0.9.1/conf
> ~~~~~~~~~~~~~~~~~~~~
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/flink/runtime/jobmanager/JobManager
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.runtime.jobmanager.JobManager
> at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> Could not find the main class:
> org.apache.flink.runtime.jobmanager.JobManager. Program will exit.
>
>
>
> ******* In taskmanager.sh, the following echo's
>
> echo Starting task manager on host $HOSTNAME
> echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
> pwd
> echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
> echo ${FLINK_ENV_JAVA_OPTS}
> echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
> echo $FLINK_JM_CLASSPATH
> echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
> echo $INTERNAL_HADOOP_CLASSPATHS
> echo "taskmanager ~~~~~~~~~~~~~~~~~~~~"
> echo $FLINK_CONF_DIR
> echo "~~~~~~~~~~~~~~~~~~~~"
> $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}"
> -classpath "`manglePathList
> "$FLINK_TM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
> org.apache.flink.runtime.taskmanager.TaskManager --configDir
> "$FLINK_CONF_DIR" --streamingMode "$STREAMINGMODE" > "$out" 2>&1 <
> /dev/null &
>
> ==> gave:
>
> taskmanager ~~~~~~~~~~~~~~~~~~~~~
> /users/camelia/thecluster
> taskmanager ~~~~~~~~~~~~~~~~~~~~~
>
> taskmanager ~~~~~~~~~~~~~~~~~~~~~
>
> taskmanager ~~~~~~~~~~~~~~~~~~~~~
> ::
> taskmanager ~~~~~~~~~~~~~~~~~~~~
> /users/camelia/thecluster/flink-0.9.1/conf
> ~~~~~~~~~~~~~~~~~~~~
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/flink/runtime/taskmanager/TaskManager
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.runtime.taskmanager.TaskManager
> at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> Could not find the main class:
> org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.
>
>
>
> ******* In flink , the following echo's
>
> echo "flink ~~~~~~~~~~~~~~~~~~~~~"
> pwd
> echo "flink ~~~~~~~~~~~~~~~~~~~~~"
> echo $CC_CLASSPATH
> echo "flink ~~~~~~~~~~~~~~~~~~~~~"
> echo $INTERNAL_HADOOP_CLASSPATHS
> echo "~~~~~~~~~~~~~~~~~~~~"
> $JAVA_RUN $JVM_ARGS "${log_setting[@]}" -classpath "`manglePathList
> "$CC_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
> org.apache.flink.client.CliFrontend "$@"
>
>
> ==> gave:
>
> flink ~~~~~~~~~~~~~~~~~~~~~
> /users/camelia/thecluster
> flink ~~~~~~~~~~~~~~~~~~~~~
>
> /users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar
> flink ~~~~~~~~~~~~~~~~~~~~~
> ::
> ~~~~~~~~~~~~~~~~~~~~
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/flink/client/CliFrontend
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.client.CliFrontend
> at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> Could not find the main class: org.apache.flink.client.CliFrontend.
> Program will exit.
>
>
> I hope that now you have a better image of the issue and I hope you can
> provide some guidance to solve it.
> I mention that I use the simplest scenario possible, just 2 nodes (1
> master and 1 slave) without Hadoop for this test.
> Also, I have a hostname command inside the script that shows me that I'm
> on the master node when running these scripts.
>
> Thank you very much!
> Camelia
>
>
> ------------------------------
> *From:* Camelia Elena Ciolac
> *Sent:* Wednesday, November 11, 2015 9:01 AM
> *To:* user@flink.apache.org
> *Subject:* RE: Cluster installation gives java.lang.NoClassDefFoundError
> for everything
>
> Good morning,
>
> Thank you Stephan!
>
> I keep on testing and in the meantime I'm wondering if the Java version on
> the cluster may be part of the issue:
> java version "1.6.0_36"
> OpenJDK Runtime Environment (IcedTea6 1.13.8) (rhel-1.13.8.1.el6_7-x86_64)
> OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
> ?
>
> The cluster has shared network file system, so no problem in this aspect.
> I keep on trying and I've put some echo's for debugging in the Flink start
> scripts.
>
> Best regards,
> Camelia
>
>
>
> ------------------------------
> *From:* Camelia Elena Ciolac
> *Sent:* Monday, November 09, 2015 2:32 PM
> *To:* user@flink.apache.org
> *Subject:* Cluster installation gives java.lang.NoClassDefFoundError for
> everything
>
> Hello,
>
> I am configuring Flink to run on a cluster with NFS.
> I have the Flink 0.9.1 distribution in some path in NFS and I added that
> path in ~/.bashrc as FLINK_HOME, and also included the $FLINK_HOME/lib
> folder to $PATH.
> I have the slaves file and the yaml file configured correctly with the
> nodes involved.
>
> Still, when I run the following command I get the following errors:
>
> $FLINK_HOME/bin/start-cluster.sh
>
> (tried also
> cd $FLINK_HOME
> ./bin/start-cluster.sh)
>
> --> in /log/flink-theuser-jobmanager-thenode.out
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/flink/runtime/jobmanager/JobManager
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.runtime.jobmanager.JobManager
> at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> Could not find the main class:
> org.apache.flink.runtime.jobmanager.JobManager. Program will exit.
>
> -->in /log/flink-theuser-taskmanager-theslavenode.out
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/flink/runtime/taskmanager/TaskManager
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.runtime.taskmanager.TaskManager
> at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> Could not find the main class:
> org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.
>
>
> And lastly, in my script I have the run of a job which gives:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/flink/client/CliFrontend
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.client.CliFrontend
> at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> Could not find the main class: org.apache.flink.client.CliFrontend.
> Program will exit.
>
> What could be the cause and which is the solution?
>
> Looking forward to your answer as soon as possible.
> Many thanks,
> Camelia
>
>
