Re: Tuple project method

2015-05-27 Thread Flavio Pompermaier
Sorry, to be effective the project should also take in input the target tuple itself :) Tuple3x,x,x reuse = tuple.project(reuse, 0,2,5)? On Wed, May 27, 2015 at 11:51 AM, Flavio Pompermaier pomperma...@okkam.it wrote: Hi flinkers, it happens very often to me that I have to output a reuse

Re: Package multiple jobs in a single jar

2015-05-27 Thread Maximilian Michels
Hi Matthias, I understand your point about advertising the interfaces but there is so much stuff to be advertised :). Honestly, I think ProgramDescription doesn't add much value although it is kind of neat. Parameters can be described in the code or by displaying a help message. However, I'm in

Tuple project method

2015-05-27 Thread Flavio Pompermaier
Hi flinkers, it happens very often to me that I have to output a reuse tuple that basically is a subset of the data contained of the input tuple..do you think it could be useful to add a project method to Tuple class? So that to be able to write something like: Tuple3x,x,x reuse =

Re: [DISCUSS] Dedicated streaming mode

2015-05-27 Thread Maximilian Michels
Hi Henry! I think the idea was to have a dedicated streaming mode as long as the default cluster mode does not support batch and streaming equally well. Once we have reached this level in the dedicated streaming mode, this will be the default cluster mode. I share your doubts about whether it is

Re: Tuple project method

2015-05-27 Thread Stephan Ewen
It would be an interesting addition. Such a method cannot be done fully type safe in Java, but that might be okay, since it is user-code internal. On Wed, May 27, 2015 at 11:52 AM, Flavio Pompermaier pomperma...@okkam.it wrote: Sorry, to be effective the project should also take in input the

Re: [DISCUSS] Canceling Streaming Jobs

2015-05-27 Thread Aljoscha Krettek
I would also prefer the second option. The first is rather a hack but not an option. :D On May 27, 2015 9:14 AM, Márton Balassi balassi.mar...@gmail.com wrote: +1 for the second option: It would also provide possibility to properly commit a state checkpoint after the terminate message was

Re: [DISCUSS] Canceling Streaming Jobs

2015-05-27 Thread Matthias J. Sax
Stephan, not sure what you mean by this exactly... But I guess, this a an add-on that can be done later. Seems to be related to https://issues.apache.org/jira/browse/FLINK-1929 I will open a JIRA for the new terminate message and assign it to myself. -Matthias On 05/27/2015 12:36 PM, Stephan

Greetings and help regarding my first issue

2015-05-27 Thread Vijendra Singh Aswal
Hi! I am new to the open source would and wanted to start my journey with Flink. I found an issue that I think would be good for me: https://issues.apache.org/jira/browse/FLINK-2077 I had a few queries regarding it and it would be great if you could help with them. 1. The issue talks about

Storm compatibility layer currently does not support Storm's SimpleJoin example

2015-05-27 Thread Szabó Péter
Hey everyone, I experimented with the Storm compatibility layer Matthias wrote, and ran some Storm examples on Flink. I found that Storm's SimpleJoin example does not work. I suppose it is because of the multiple input streams. I'm willing to add another example instead. Right now, I'm getting it

[jira] [Created] (FLINK-2098) Checkpoint barrier initiation at source is not aligned with snapshotting

2015-05-27 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-2098: --- Summary: Checkpoint barrier initiation at source is not aligned with snapshotting Key: FLINK-2098 URL: https://issues.apache.org/jira/browse/FLINK-2098 Project: Flink

[jira] [Created] (FLINK-2102) Add predict operation for LabeledVector

2015-05-27 Thread Theodore Vasiloudis (JIRA)
Theodore Vasiloudis created FLINK-2102: -- Summary: Add predict operation for LabeledVector Key: FLINK-2102 URL: https://issues.apache.org/jira/browse/FLINK-2102 Project: Flink Issue

Re: SQL on Flink

2015-05-27 Thread Ufuk Celebi
On 27 May 2015, at 17:05, Timo Walther twal...@apache.org wrote: It's rather passion for the future of the project than passion for SQL ;-) I always try to think like someone from the economy. And IMO the guys from economy are still thinking in SQL. If you want to persuade someone coming

Re: SQL on Flink

2015-05-27 Thread Kostas Tzoumas
very excited to see this starting! On Wed, May 27, 2015 at 6:06 PM, Ufuk Celebi u...@apache.org wrote: On 27 May 2015, at 17:05, Timo Walther twal...@apache.org wrote: It's rather passion for the future of the project than passion for SQL ;-) I always try to think like someone from the

Re: SQL on Flink

2015-05-27 Thread Fabian Hueske
IMO, it is better to have one feature that is reasonably well developed instead of two half-baked features. That's why I proposed to advance the Table API a bit further before starting the next big thing. I played around with the Table API recently and I think it definitely needs a bit more

Re: SQL on Flink

2015-05-27 Thread Timo Walther
It's rather passion for the future of the project than passion for SQL ;-) I always try to think like someone from the economy. And IMO the guys from economy are still thinking in SQL. If you want to persuade someone coming from the SQL world, you should offer a SQL interface to run legacy

[jira] [Created] (FLINK-2101) Scheme Inference doesn't work for Tuple5

2015-05-27 Thread Rico Bergmann (JIRA)
Rico Bergmann created FLINK-2101: Summary: Scheme Inference doesn't work for Tuple5 Key: FLINK-2101 URL: https://issues.apache.org/jira/browse/FLINK-2101 Project: Flink Issue Type: Bug

Re: SQL on Flink

2015-05-27 Thread Fabian Hueske
+1 for committer passion! Please don't get me wrong, I think SQL on Flink would be a great feature. I just wanted to make the point that the Table API needs to mirror all SQL features, if SQL is implemented on top of the Table API. 2015-05-27 16:34 GMT+02:00 Kostas Tzoumas ktzou...@apache.org:

Re: SQL on Flink

2015-05-27 Thread Kostas Tzoumas
I think Fabian's arguments make a lot of sense. However, if Timo *really wants* to start SQL on top of Table, that is what he will do a great job at :-) As usual, we can keep it in beta status in flink-staging until it is mature... and it will help create issues for the Table API and give

Re: SQL on Flink

2015-05-27 Thread Aljoscha Krettek
+1 to what ufuk said. :D On May 27, 2015 6:13 PM, Kostas Tzoumas ktzou...@apache.org wrote: very excited to see this starting! On Wed, May 27, 2015 at 6:06 PM, Ufuk Celebi u...@apache.org wrote: On 27 May 2015, at 17:05, Timo Walther twal...@apache.org wrote: It's rather passion for

Re: [DISCUSS] Canceling Streaming Jobs

2015-05-27 Thread Gyula Fóra
Hey, I would also strongly prefer the second option, users need to have the option to force cancel a program in case of something unwanted behaviour. Cheers, Gyula Matthias J. Sax mj...@informatik.hu-berlin.de ezt írta (időpont: 2015. máj. 27., Sze, 1:20): Hi, currently, the only way to

Adding custom Tuple to a DataSet

2015-05-27 Thread Amit Pawar
Hi Is there a way, where I can add a custom (newly created) Tuple to a new DataSet or already existing DataSet? DataSet set = env.fromElements (myCustomTuple); works fine, but only with same datatype in case of Tuple2 or higher. Tuple2String,Long creates a problem (as stated in JavaDoc it needs