Re: design question

2016-04-24 Thread Chen Bekor
cool - can you point me to some docs about how to configure Rocks DB? I searched the online docs and found nothing substantial. Also - If I'm using HDFS (S3backed ) cluster, how would that effect RocksDB? can I configure it to run on optimized SSD etc? any help is appreciated. On Sun, Apr 24,

General Data questions - streams vs batch

2016-04-24 Thread Konstantin Kulagin
Hi guys, I have some kind of general question in order to get more understanding of stream vs final data transformation. More specific - I am trying to understand 'entities' lifecycle during processing. 1) For example in a case of streams: suppose we start with some key-value source, parallel it

Re: Flink on Yarn - ApplicationMaster command

2016-04-24 Thread Theofilos Kakantousis
Hi, The issue was a mismatch of jar versions on my client. Seems to be working fine now. Thanks again for your help! Cheers, Theofilos On 2016-04-22 18:22, Theofilos Kakantousis wrote: Hi Max, I manage to get the jobManagerAddress from FlinkYarnCluster, however when I submit a job using

Re: "No more bytes left" at deserialization

2016-04-24 Thread Timur Fayruzov
Hello Robert, I'm on 1.0.0 compiled with Scala 2.11. The second exception was an issue with a cluster (that I didn't dig into), when I restarted the cluster I was able to go past it, so now I have the following exception: java.lang.Exception: The data preparation for task 'CHAIN CoGroup (CoGroup

Re: "No more bytes left" at deserialization

2016-04-24 Thread Timur Fayruzov
Trying to use ProtobufSerializer -- program consistently fails with the following exception: java.lang.IllegalStateException: Update task on instance 52b9d8ffa1c0bb7251a9731c565460eb @ ip-10-105-200-137 - 1 slots - URL: akka.tcp://flink@10.105.200.137:48990/user/taskmanager failed due to: at

"No more bytes left" at deserialization

2016-04-24 Thread Timur Fayruzov
Hello, I'm running a Flink program that is failing with the following exception: 2016-04-23 02:00:38,947 ERROR org.apache.flink.client.CliFrontend - Error while running the command. org.apache.flink.client.program.ProgramInvocationException: The program execution failed: Job