The slot in which the task was scheduled has been killed (probably loss of TaskManager)

Hi to all, I have this strange error in my job and I don't know what's going on. What can I do? The full exception is: The slot in which the task was scheduled has been killed (probably loss of TaskManager). at org.apache.flink.runtime.instance.SimpleSlot.cancel(SimpleSlot.java:98) at

Re: JobTimeoutException: Lost connection to JobManager

The exception indicates that you're still using the old version. It takes some time for the new Maven artifact to get deployed to the snapshot repository. Apparently, a artifact has already been deployed this morning. Did you delete the jar files in your .m2 folder? On Wed, Apr 15, 2015 at 1:38

Orphaned chunks

Hi to all, another error today :( My job ended with a lot of Orphaned chunk of bytes found during finalize. What could be the cause of this error? Best, Flavio

Re: JobTimeoutException: Lost connection to JobManager

On 15 Apr 2015, at 14:18, Maximilian Michels m...@apache.org wrote: The exception indicates that you're still using the old version. It takes some time for the new Maven artifact to get deployed to the snapshot repository. Apparently, a artifact has already been deployed this morning. Did

Re: Is there a data type stores name filed and datatype of field and return field by name

The table API (see http://ci.apache.org/projects/flink/flink-docs-master/table.html) is exactly for that. Check it out! On Wed, Apr 15, 2015 at 4:23 PM, hagersaleh loveallah1...@yahoo.com wrote: Is there a data type stores name filed and datatype of field and return field by name i want

Re: Orphaned chunks

Hey Flavio, I was not able to find the String Orphaned chunk in the Flink code base. However, I found it here: https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/memory/GlobalMemoryManager.java#L157 Maybe you've send the message to the wrong mailing list?

Re: Orphaned chunks

Yes Robert, Unfortunately I discovered that the error was caused by Phoenix just a little bit later the mail sending. The error is generated in the finalize() method of Pheonix MemoryManager so it seems somehow related to gc. I rerun the experiment logging to a file so I can investigate deeper

taskmanager.network.numberOfBuffers

Hi to all, I've received an error running the job saying to increase this parameter so I set it to 2048*4 and everything worked. However, could you explain me in detail how this number is computed? I'm running the job from my IDE (default parallelism so all my 8 cores) so I was expecting no such

Re: taskmanager.network.numberOfBuffers

On 15 Apr 2015, at 09:37, Flavio Pompermaier pomperma...@okkam.it wrote: I've received an error running the job saying to increase this parameter so I set it to 2048*4 and everything worked. However, could you explain me in detail how this number is computed? I'm running the job from my IDE

Re: Left outer join

On 15 Apr 2015, at 10:30, Flavio Pompermaier pomperma...@okkam.it wrote: Hi to all, I have to join two datasets but I'd like to keep all data in the left also if there' no right dataset. How can you achieve that in Flink? maybe I should use coGroup? Yes, currently you have to implement

Re: Left outer join

Do you have an already working example of it? :) On Wed, Apr 15, 2015 at 10:32 AM, Ufuk Celebi u...@apache.org wrote: On 15 Apr 2015, at 10:30, Flavio Pompermaier pomperma...@okkam.it wrote: Hi to all, I have to join two datasets but I'd like to keep all data in the left also if there'

Re: Left outer join

please add link to explain left join using cogroup or add example very thanks -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Left-outer-join-tp1031p1034.html Sent from the Apache Flink User Mailing List archive. mailing list archive at

Re: Left outer join

I think this may be a great example to add as a utility function. Or actually add as an function to the DataSet, internally realized as a special case of coGroup. We do not have a ready example of that, but it should be straightforward to realize. Similar as for the join, coGroup on the join

Google Compute Engine Cluster

Hi guys, I'm trying to setup a simple Flink cluster on Google Compute Engine. I'm running 3 nodes (1 master, 2 workers). On master node I set up ssh key and moved it into authorized_keys. When I try to copy my key to each worker node I got Permission denied (publickey). Someone had the same

Left outer join

Hi to all, I have to join two datasets but I'd like to keep all data in the left also if there' no right dataset. How can you achieve that in Flink? maybe I should use coGroup? Best, Flavio

Re: Left outer join

Hi Flavio, Here's an simple example of a Left Outer Join: https://gist.github.com/mxm/c2e9c459a9d82c18d789 As Stephan pointed out, this can be very easily modified to construct a Right Outer Join (just exchange leftElements and rightElements in the two loops). Here's an excerpt with the most