Re: Flink on Yarn delegation token renewal

2016-11-08 Thread Theofilos Kakantousis
Am 08.11.2016 um 10:25 schrieb Theofilos Kakantousis <t...@kth.se <mailto:t...@kth.se>>: Hi everyone, I'm using Flink 1.1.3 with Hadoop 2.7.3 and was wondering about delegation token renewal when running Flink on Yarn. Yarn demands services to renew delegation tokens

Flink on Yarn delegation token renewal

2016-11-08 Thread Theofilos Kakantousis
Hi everyone, I'm using Flink 1.1.3 with Hadoop 2.7.3 and was wondering about delegation token renewal when running Flink on Yarn. Yarn demands services to renew delegation tokens on their own and if not, Yarn jobs will fail after one week.

Re: Application log on Yarn FlinkCluster

2016-06-15 Thread Theofilos Kakantousis
correctly including the TaskManager logs. I'm wondering, is there anything in the Hadoop logs of the Resourcemanager/Nodemanager that could indicate a transfer failure? On Wed, Jun 15, 2016 at 10:18 AM, Theofilos Kakantousis <t...@kth.se <mailto:t...@kth.se>> wrote: Hi,

Re: Application log on Yarn FlinkCluster

2016-06-15 Thread Theofilos Kakantousis
. Cheers, Theofilos On 6/15/2016 10:14 AM, Maximilian Michels wrote: Please use the `yarn logs -applicationId ` to retrieve the logs. If you have enabled log aggregation, this will give you all container logs concatenated. Cheers, Max On Wed, Jun 15, 2016 at 12:24 AM, Theofilos Kakantousis &l

Re: Application log on Yarn FlinkCluster

2016-06-14 Thread Theofilos Kakantousis
ith the network communication if the Client doesn't return from `runBlocking(..)`. Would be interesting to take a look at the logs to find out why that could be. Cheers, Max On Sat, Jun 11, 2016 at 1:53 PM, Theofilos Kakantousis <t...@kth.se> wrote: Hi Robert, Thanks for the prompt reply

Re: Application log on Yarn FlinkCluster

2016-06-11 Thread Theofilos Kakantousis
? (I'm asking since different parts are executed on the cluster and locally). On Fri, Jun 10, 2016 at 4:00 PM, Theofilos Kakantousis <t...@kth.se <mailto:t...@kth.se>> wrote: Hi all, Flink 1.0.3 Hadoop 2.4.0 When running a job on a Flink Cluster on Yarn, the

Application log on Yarn FlinkCluster

2016-06-10 Thread Theofilos Kakantousis
Hi all, Flink 1.0.3 Hadoop 2.4.0 When running a job on a Flink Cluster on Yarn, the application output is not included in the Yarn log. Instead, it is only printed in the stdout from where I run my program. For the jobmanager, I'm using the log4j.properties file from the flink/conf

Re: Streaming job on YARN - ClassNotFoundException

2016-05-06 Thread Theofilos Kakantousis
? Thanks, Theo On 2016-05-06 19:50, Robert Metzger wrote: Hi Theo, you can't mix different Flink versions in your dependencies. Please use 1.0.2 for the flink_yarn client as well or 1.1-SNAPSHOT everywhere. On Fri, May 6, 2016 at 7:02 PM, Theofilos Kakantousis <t...@kth.se <mailto:t...@

Streaming job on YARN - ClassNotFoundException

2016-05-06 Thread Theofilos Kakantousis
Hi everyone, Flink 1.0.2 Hadoop 2.4.0 I am running Flink on Yarn by using FlinkYarnClient to launch a Flink cluster and Flink Client to submit a PackagedProgram. To keep it simple, for batch jobs I use the WordCount example and for streaming the IterateExample and IncrementalLearning ones

Re: Flink Client use remote app jar

2016-04-28 Thread Theofilos Kakantousis
available. But we could maybe add it as a tool which we contribute to flink-contrib. Do you wanna take the lead? Cheers, Till On Wed, Apr 27, 2016 at 10:12 AM, Theofilos Kakantousis <t...@kth.se <mailto:t...@kth.se>> wrote: Hi Till, Thank you for the quick reply. Do you thin

Re: Flink Client use remote app jar

2016-04-27 Thread Theofilos Kakantousis
26, 2016 11:54 PM, "Theofilos Kakantousis" <t...@kth.se <mailto:t...@kth.se>> wrote: Hi everyone, Flink 0.10.1 Hadoop 2.4.0 Fairly new to Flink here, so my question might be simple but couldn't find something relevant in the docs. I am implement

Re: Flink on Yarn - ApplicationMaster command

2016-04-24 Thread Theofilos Kakantousis
Hi, The issue was a mismatch of jar versions on my client. Seems to be working fine now. Thanks again for your help! Cheers, Theofilos On 2016-04-22 18:22, Theofilos Kakantousis wrote: Hi Max, I manage to get the jobManagerAddress from FlinkYarnCluster, however when I submit a job using

Flink on Yarn - ApplicationMaster command

2016-04-19 Thread Theofilos Kakantousis
Hi everyone, I'm using Flink 0.10.1 and hadoop 2.4.0 to implement a client that submits a flink application to Yarn. To keep it simple I use the ConnectedComponents app from flink examples. I set the required properties (Resources, AM ContainerLaunchContext etc.) on the YARN client