Re: Class loading issues when using Remote Execution Environment

Chesnay, I have filed https://issues.apache.org/jira/browse/FLINK-9267 to keep track of this issue. Regards, Kedar On Fri, Apr 27, 2018 at 11:50 AM, kedar mhaswade <kedar.mhasw...@gmail.com> wrote: > Thanks again! > > This is strange. With both Flink 1.3.3 and Flink 1.6.0-S

Re: Class loading issues when using Remote Execution Environment

etter to use jarFiles argument on > createRemoteEnvironment? *Yes, once we get it working this is the way to > go. > > > On 26.04.2018 18:42, kedar mhaswade wrote: > > Thanks Chesnay for your incredible help! > > I will try out the suggestions again. A few questions: > - What ve

Re: Setting the parallelism in a cluster of machines properly

On Thu, Apr 26, 2018 at 10:47 AM, Makis Pap wrote: > OK Michael! > > I will look into it and will come back at you! Thanks for the help. I > agree that it is quite suspicious the par = 8 > > Jps? Meaning? > jps is a tool that comes with JDK (see $JAVA_HOME/bin). This is

Re: Class loading issues when using Remote Execution Environment

s into lib (but my guess > is you missed a gradoop jar) > > Note that the job fails to run since you use "LocalCollectionOutputFormat" > which can only be used for local execution, i.e. when the job submission > and execution happen in the same JVM. > > > On 25.0

Re: Class loading issues when using Remote Execution Environment

hesnay Schepler <ches...@apache.org> wrote: > I couldn't spot any error in what you tried to do. Does the job-submission > succeed if you submit the jar through the command-line client? > > Can you share the project, or a minimal reproducing version? > > > On 25.04.2018 00:

Class loading issues when using Remote Execution Environment

I am trying to get gradoop_demo (a gradoop based graph visualization app) working on Flink with *Remote* Execution Environment. This app, which is based on Gradoop, submits a job to the *preconfigured* execution environment, collects the results and

Re: Flink 1.4.2 in Zeppelin Notebook

er compiled against flink > 1.1.3. > When submitting a job with the CLI tool everything is working fine. The > CLI tool is version 1.4.2 ... > > Any other suggestions? > > Thanks a lot. > Best, > Rico. > > > Am 06.04.2018 um 18:44 schrieb kedar mhaswade &l

Re: Flink 1.4.2 in Zeppelin Notebook

Yes. You need to add the two properties for the job manager (I agree, it is confusing because the properties named "host" and "port" already available, but the names of the useful properties are different): Could you please try this and let us know if it works for you? Regards, Kedar On Fri,

Re: Flink TaskManager and JobManager internals

On Wed, Mar 28, 2018 at 3:14 AM, Niclas Hedhman wrote: > Hi, > > is there some document (or presentation) that explains the internals of > how a Job gets deployed on to the cluster? Communications, Classloading and > Serialization (if any) are the key points here I think. > I

Programmatic creation of YARN sessions and deployment (running) Flink jobs on it.

Typically, when one wants to run a Flink job on a Hadoop YARN installation, one creates a Yarn session (e.g. ./bin/yarn-session.sh -n 4 -qu test-yarn-queue) and runs intended Flink job(s) (e.g. ./bin/flink run -c MyFlinkApp -m job-manager-host:job-manager-port myapp.jar) on the Flink cluster

Re: [DISCUSS] Inverted (child-first) class loading

-master/monitoring/debugging_classloading.html > > Best, > Aljoscha > > > On 10. Mar 2018, at 05:53, kedar mhaswade <kedar.mhasw...@gmail.com> > wrote: > > This is an interesting question and it usually has consequences that are > far-reaching in user experience. > > If a F

Running the executables from Flink distribution of the source build

Flink gurus! I have built Flink from source. I find that the executables are all at: /flink-dist/target/flink-1.6-SNAPSHOT-bin/flink-1.6-SNAPSHOT. However, when I try to run start-scala-shell.sh local from the bin subfolder of this folder, it does not seem to run the simple wordcount example.

Re: [DISCUSS] Inverted (child-first) class loading

This is an interesting question and it usually has consequences that are far-reaching in user experience. If a Flink app is supposed to be a "standalone app" that any Flink installation should be able to run, then the child-first classloading makes sense. This is how we build many of the Java

Re: Job is be cancelled, but the stdout log still prints

Also, in addition to what Gary said, if you take Flink completely out of picture and wrote a simple Java class with a main method and the static block (!) which does some long running task like getLiveInfo(), then chances are that your class will make the JVM hang! Basically what you are doing is

CliFrontend hang in the local case?

I am seeing a hang where the main thread of CliFrontend goes to timed waiting. This appears like a livelock. My local setup is simple: A job manager, a task manager on MacOS. My algorithm is based on Gelly's vertex centric computation. The resultant graph's vertex count is about 4 million. I am