Re: Sorting of fields

2015-02-04 Thread Timo Walther
Ok, I found an earlier discussion about it. Sorry for the mail. However, I think this is a very important feature and I should be added soon. On 04.02.2015 14:38, Timo Walther wrote: Hey, is it correct that we currently do not support sorting without any grouping? I had this question by 2

Re: Eclipse JDT, Java 8, lambdas

2015-02-09 Thread Timo Walther
Hey, it seems that 4.4.2 also includes the fix (https://projects.eclipse.org/projects/eclipse/releases/4.4.2/bugs) and will be released end of february. I will try Eclipse Luna SR2 RC2 today and check if it is working. Regards, Timo On 09.02.2015 10:05, Nam-Luc Tran wrote: I did try the

Re: Eclipse JDT, Java 8, lambdas

2015-02-11 Thread Timo Walther
... On 09.02.2015 16:28, Stephan Ewen wrote: Is it possible to use this compiler for the java 8 quickstart archetypes? On Mon, Feb 9, 2015 at 4:14 PM, Timo Walther twal...@apache.org wrote: The fix is included in 4.4.2. However, it seems that even if the compiler option

Re: kryoException : Buffer underflow

2015-02-11 Thread Timo Walther
, 2015 at 9:28 PM, Timo Walther twal...@apache.org wrote: Hey Nam-Luc, I think your problem lies in the following line: .returns(eu.euranova.flink.Centroid25) If you do not specify the fields of the class in the String by using myfield=String,otherField=int, the underlying parser will create

Re: kryoException : Buffer underflow

2015-02-11 Thread Timo Walther
Hey Nam-Luc, I think your problem lies in the following line: .returns(eu.euranova.flink.Centroid25) If you do not specify the fields of the class in the String by using myfield=String,otherField=int, the underlying parser will create an GenericTypeInfo type information which then uses Kryo

Re: Fwd: TypeSerializerInputFormat cannot determine its type automatically

2015-01-29 Thread Timo Walther
Hey Alexander, I have looked into your issue. You can simply use env.createInput(InputFormat,TypeInformation) instead of env.readFile() then you can pass TypeInformation manually without implementing ResultTypeQueryable. Regards, Timo On 29.01.2015 14:54, Alexander Alexandrov wrote: The

Re: Semantic Properties and Functions with Iterables

2015-03-09 Thread Timo Walther
is should be made clear very soon. I will open a JIRA for that. Thanks, Fabian 2015-03-06 10:19 GMT+01:00 Timo Walther twal...@apache.org: Hey all, I'm currently working a lot on the UDF static code analyzer. But I have a general question about Semantic Properties which might be also

Semantic Properties and Functions with Iterables

2015-03-06 Thread Timo Walther
Hey all, I'm currently working a lot on the UDF static code analyzer. But I have a general question about Semantic Properties which might be also interesting for other users. How is the ForwardedFields annotation interpreted for UDF functions with Iterables? An example can be found in:

Re: [VOTE] Name of Expression API Representation

2015-03-26 Thread Timo Walther
+Table API Same thoughts as Stephan. Table is more common in the economy than Relation. On 25.03.2015 21:30, Stephan Ewen wrote: +Table API / Table I have a feeling that Relation is a name mostly used by people with a deeper background in (relational) databases, while table is more the

Re: [DISCUSS] Make a release to be announced at ApacheCon

2015-03-26 Thread Timo Walther
+1 for a beta release. So there is no feature-freeze until the RC right? On 26.03.2015 15:32, Márton Balassi wrote: +1 for the early release. I'd call it 0.9-milestone1. On Thu, Mar 26, 2015 at 1:37 PM, Maximilian Michels m...@apache.org wrote: +1 for a beta release: 0.9-beta. On Thu, Mar

Re: Merge Python API

2015-04-20 Thread Timo Walther
+1 On 20.04.2015 14:49, Gyula Fóra wrote: +1 On Mon, Apr 20, 2015 at 2:41 PM, Fabian Hueske fhue...@gmail.com wrote: +1 2015-04-20 14:39 GMT+02:00 Maximilian Michels m...@apache.org: +1 Let's merge it to flink-staging and get some people to use it. On Mon, Apr 20, 2015 at 2:21 PM, Kostas

Re: New project website

2015-05-11 Thread Timo Walther
I completely agree with Robert. Less marketing, more technical information targeted to the user group. I think the diagram could be a little bit larger and the blog post smaller, so that the lines meet in the middle ;) On 11.05.2015 14:39, Robert Metzger wrote: I think its fine when the

Re: About Operator and OperatorBase

2015-04-16 Thread Timo Walther
I share Stephans opinion. By the way, we could also find a common name for operators with two inputs. Sometimes it's TwoInputXXX, DualInputXXX, BinaryInputXXX... pretty inconsistent. On 15.04.2015 17:48, Till Rohrmann wrote: I would also be in favour of making the distinction between the

Re: SQL on Flink

2015-05-27 Thread Timo Walther
this. Since the Table API is still a fairly new component and not very feature rich, it might make sense to extend and strengthen it before putting something major on top. Cheers, Fabian 2015-05-26 21:38 GMT+02:00 Timo Walther twal...@apache.org: Hey everyone, I would be interested in having

Re: Function input type validation

2015-11-08 Thread Timo Walther
The reason for input validation is to check if the Function is fully compatible. Actually only the return types are necessary, but it prohibits stupid implementation mistakes and undesired behavior. E.g. if you implement a "class MyMapper extends MapFunction{}" and use it for

Re: Error in during TypeExtraction

2015-11-12 Thread Timo Walther
This looks like a bug. Can you open an issue for that? I will look into it later. Regards, Timo On 12.11.2015 13:16, Gyula Fóra wrote: Hey, I get a weird error when I try to execute my job on the cluster. Locally this works fine but running it from the command line fails during

Re: [DISCUSSION] Type hints versus TypeInfoParser

2015-11-11 Thread Timo Walther
owerful and type safe than the String/Parser way of giving hints, I was wondering whether we should add this and deprecate the String variant. If we do that, 1.0 is the time to do that. What do you think about this idea? @Timo Walther Since you worked a lot on types/parser/etc - what is your

Re: Powered by Flink

2015-10-19 Thread Timo Walther
+1 for adding it to the website instead of wiki. "Who is using Flink?" is always a question difficult to answer to interested users. On 19.10.2015 15:08, Suneel Marthi wrote: +1 to this. On Mon, Oct 19, 2015 at 3:00 PM, Fabian Hueske > wrote:

Re: Powered by Flink

2015-10-19 Thread Timo Walther
Ah ok, sorry. I think linking to the wiki is also ok. On 19.10.2015 15:18, Fabian Hueske wrote: @Timo: The proposal was to keep the list in the wiki (can be easily extended) but link from the main website to the wiki page. 2015-10-19 15:16 GMT+02:00 Timo Walther <twal...@apache.org>

Add BigDecimal and BigInteger as types

2015-11-18 Thread Timo Walther
Hey everyone, I'm not sure if we already had a discussion about it but as we are currently adding new types like the Either type, I would like to discuss it again. I think especially for business or scientific applications it makes sense to support the BigInteger and BigDecimal types

Re: [DISCUSSION] Type hints versus TypeInfoParser

2015-11-18 Thread Timo Walther
If the TypeExtractor is not able to handle the fields of a Pojo correctly, the String parser is quite useful to say

Re: [ANNOUNCE] Welcome Matthias Sax as new committer

2015-09-02 Thread Timo Walther
Congratulations Matthias! Regards, Timo On 02.09.2015 13:32, Chiwan Park wrote: Welcome Matthias! :) Regards, Chiwan Park On Sep 2, 2015, at 8:30 PM, Kostas Tzoumas wrote: The Project Management Committee (PMC) of Apache Flink has asked Matthias Sax to become a

Re: Either not NotSerializableException and InvalidTypesException

2015-11-29 Thread Timo Walther
Hi Vasia, regarding your TypeExtractor problem. The TypeExtractor works correctly. The with() function of the JoinOperator calls the wrong TypeExtractor method that does not allow missing type info. This is a bug. Can open an issue for that? Regards, Timo On 28.11.2015 20:18, Vasiliki

Re: The null in Flink

2015-11-25 Thread Timo Walther
Hi Chengxiang, I totally agree that the Table API should fully support NULL values. The Table API is a logical API and therefore we should be as close to ANSI SQL as possible. Rows need to be nullable in the near future. 2. i, ii, iii and iv sound reasonable. But v, vi and vii sound to much

Re: withParameters() for Streaming API

2015-11-24 Thread Timo Walther
. Sax wrote: We had this discussion a while ago. If I recall correctly, "withParameters()" is not encourage to be used in DataSet either. This is the thread: https://mail-archives.apache.org/mod_mbox/flink-dev/201509.mbox/%3C55EC69CD.1070003%40apache.org%3E -Matthias On 11/24/2015 02:1

withParameters() for Streaming API

2015-11-24 Thread Timo Walther
Hi all, I want to set the Configuration of a streaming operator and access it via the open method of the RichFunction. There is no possibility to set the Configuration of the open method at the moment, right? Can I open an issue for a withParameters() equivalent for the Stremaing API?

Re: [DISCUSSION] Type hints versus TypeInfoParser

2015-11-19 Thread Timo Walther
. (Not sure if all that is implemented in the TypeExtractor, though). What do you think? Stephan On Wed, Nov 18, 2015 at 6:03 PM, Timo Walther <twal...@apache.org> wrote: If the TypeExtractor is not able to handle the fields of a Pojo correctly, the String parser is quite useful to say "org.my.Pojo

Re: Add BigDecimal and BigInteger as types

2015-11-19 Thread Timo Walther
. They are currently generic types and handled by Kryo, which has (AFAIK) proper serializers for them. Are there more benefits of native support (other than more compact serialization) that you are thinking of? On Wed, Nov 18, 2015 at 5:55 PM, Timo Walther <twal...@apache.org> wrote: Hey everyone

Re: primitiveDefaultValue in CodeGenUtils in Table API

2016-06-29 Thread Timo Walther
Hi Cody, default values are needed in cases where NULL values are not supported. This happens if the null check is disabled in TableConfig for efficiency reasons. Using 0 to DataType.MAX_VALUE for numeric types and -1 as a NULL equivalent in special cases seems more reasonable to me. Hope

Re: Table API / SQL Queries and Code Generation

2016-04-05 Thread Timo Walther
Hi Gábor, the code generation in the Table API is in a very early stage and contains not much optimization logic so far. Currently we extend the functionality to support the most important SQL operations. It will need some time until we can further optimize the generated code (e.g. for

Re: Input type validation is killing me

2016-03-02 Thread Timo Walther
Can you open an issue with an example of your custom TypeInfo? I will then open a suitable PR for it. On 02.03.2016 15:33, Gyula Fóra wrote: Would that work with generic classes? Timo Walther <twal...@apache.org> ezt írta (időpont: 2016. márc. 2., Sze, 15:22): After thinking about

Re: Input type validation is killing me

2016-03-02 Thread Timo Walther
:00, Gyula Fóra wrote: Hi! Yes I think, that sounds good :) We just need to make sure that this works with things like the TupleTypeInfo which is built-on but I can still mix in new Types for the fields. Thanks, Gyula Timo Walther <twal...@apache.org> ezt írta (időpont: 2016. márc. 2., S

Re: Use case

2016-07-28 Thread Timo Walther
this be enabled? My iterative stream is also quite small: only one value. Is there a better way to keep track of a largest value in a stream? Regards, Kevin -- Freundliche Grüße / Kind Regards Timo Walther Follow me: @twalthr https://www.linkedin.com/in/twalthr

Re: Flink Table & SQL doesn't work in very simple example

2016-07-20 Thread Timo Walther
-tableXXX.jar already includes everyting it needs (calcite, avative, eigenbase, etc.). Hope that helps. Am 20/07/16 um 14:14 schrieb Timo Walther: You can always find the latest nightly snapshot version here: http://flink.apache.org/contribute-code.html (at the end of the page) Am 20/07/16 um 14:08

Re: Flink Table & SQL doesn't work in very simple example

2016-07-20 Thread Timo Walther
t there is some overlapping between libraries that contain the ImmutableList type? google/common/collect/ImmutableList (with flink shaded)? As per the error "/apache/flink/shaded/calcite/com/google/common/collect/ImmutableList;" -Original Message- From: Maximilian Michels [mailto:m..

Re: Flink Table & SQL doesn't work in very simple example

2016-07-20 Thread Timo Walther
an it be that there is some overlapping between libraries that contain the ImmutableList type? google/common/collect/ImmutableList (with flink shaded)? As per the error "/apache/flink/shaded/calcite/com/google/common/collect/ImmutableList;" -Original Message- From: Maximilian Mich

Re: [DISCUSS] Table API / SQL indicators for event and processing time

2017-02-22 Thread Timo Walther
sender by phone or email immediately and delete it! -Original Message- From: Timo Walther [mailto:twal...@apache.org] Sent: Wednesday, February 15, 2017 9:33 AM To: dev@flink.apache.org Subject: Re: [DISCUSS] Table API / SQL indicators for event and processing time Hi all, at first I als

Re: Contribute to Flink

2017-02-20 Thread Timo Walther
Welcome to the Flink community, Jin! I gave you contributor permissions, so you can assign issues to yourself. Regards, Timo Am 20/02/17 um 14:47 schrieb Jin Mingjian: Hi, Flink dev community, I'd like to contribute to Flink. Particularly, I am interested in kinds of optimization works in

Re: [DISCUSS] Table API / SQL indicators for event and processing time

2017-02-15 Thread Timo Walther
Hi all, at first I also thought that built-in functions (rowtime() and proctime()) are the easiest solution. However, I think to be future-proof we should make them system attributes; esp. to relate them to a corresponding table in case of multiple tables. Logically they are attributes of

Re: New Flink team member - Kate Eri.

2017-01-17 Thread Timo Walther
Hi Katherin, great to hear that you would like to contribute! Welcome! I gave you contributor permissions. You can now assign issues to yourself. I assigned FLINK-1750 to you. Right now there are many open ML pull requests, you are very welcome to review the code of others, too. Timo Am

Re: <无主题>

2016-08-16 Thread Timo Walther
Hi Renkai, thanks for the interest in Flink and the Table API/SQL! I would recommend to maybe have a look into previous Pull Requests for new SQL features such as INTERSECT [1] to get an overview of layers and classes. I also recommend the design document [2] that describes the all planned

Re: [DISCUSS] Table API / SQL indicators for event and processing time

2017-03-01 Thread Timo Walther
Int, String, Long)]("MyTable", 'a, 'b, 'c) .registeredWatermarks('a, waterMarkFunction1) SELECT a, SUM(b) OVER (PARTITION BY c ORDER BY a RANGE BETWEEN 2 PRECEDING) AS sumB FROM MyTable What do you think ? 2017-02-22 23:44 GMT+08:00 Timo Walther <twal...@apache.org>

Re: [DISCUSS] Some thoughts about unify Stream SQL and Batch SQL grammer

2016-08-23 Thread Timo Walther
SQL. Best, Fabian [1] http://calcite.apache.org/docs/stream.html#tumbling-windows-improved [2] https://issues.apache.org/jira/browse/CALCITE-1345 -- Freundliche Grüße / Kind Regards Timo Walther Follow me: @twalthr https://www.linkedin.com/in/twalthr

Re: [DISCUSS] FLIP-11: Table API Stream Aggregations

2016-09-06 Thread Timo Walther
aggregates (SlideRow unboundedPreceding). Can you have a look at the FLIP and give feedback whether this is what you are looking for? Improvement suggestions are very welcome as well. Thank you, Fabian 2016-09-01 16:12 GMT+02:00 Timo Walther <twal...@apache.org>: Hi all! Fabian and I worked

Re: Extending FLIP template

2016-09-01 Thread Timo Walther
steps or subtask the FLIP can be implemented. The described subtasks will then be reflected as individual JIRA issues. What do you think? Cheers, Fabian -- Freundliche Grüße / Kind Regards Timo Walther Follow me: @twalthr https://www.linkedin.com/in/twalthr

[DISCUSS] FLIP-11: Table API Stream Aggregations

2016-09-01 Thread Timo Walther
Hi all! Fabian and I worked on a FLIP for Stream Aggregations in the Table API. You can find the FLIP-11 here: https://cwiki.apache.org/confluence/display/FLINK/FLIP-11%3A+Table+API+Stream+Aggregations Motivation for the FLIP: The Table API is a declarative API to define queries on static

Re: [DISCUSS] Some thoughts about unify Stream SQL and Batch SQL grammer

2016-08-30 Thread Timo Walther
think the TableSourceITCase.testCsvTableSource in batch and stream package can explain it. Am I right ? - Jark Wu 在 2016年8月29日,下午8:59,Timo Walther <twal...@apache.org> 写道: At first glance, I thought we are losing the possibility to distingish between choosing a batch or streaming

Re: KafkaProducer can not be instantiated

2016-10-06 Thread Timo Walther
lone fix. Cheers, Gordon On October 5, 2016 at 10:51:05 PM, Timo Walther (twal...@apache.org) wrote: Hey everyone, I'm currently rewriting the KafkaTabeSinkTest and discovered something that doesn't seem to be intended: Is it intended that FlinkKafkaProducer08 cannot be instantiated wit

KafkaProducer can not be instantiated

2016-10-05 Thread Timo Walther
Hey everyone, I'm currently rewriting the KafkaTabeSinkTest and discovered something that doesn't seem to be intended: Is it intended that FlinkKafkaProducer08 cannot be instantiated without a running Kafka instance? The constructor of FlinkKafkaProducerBase calls getKafkaProducer() which

Re: Some questions about Table API and FlinkSQL

2016-10-04 Thread Timo Walther
UnitCompiler.java:7476) In this time if I am execute for int column ('_2') i getting result is equals '0.0' What am I doing wrong? Best regards, Anton Mushin -- Freundliche Grüße / Kind Regards Timo Walther Follow me: @twalthr https://www.linkedin.com/in/twalthr

Re: [DISCUSS] FLIP-11: Table API Stream Aggregations

2016-09-19 Thread Timo Walther
ime object systemtime What do you think about this? - Jark Wu 在 2016年9月6日,下午11:00,Timo Walther <twal...@apache.org <mailto:twal...@apache.org>> 写道: Hi all, I thought about the API of the FLIP again. If we allow the "systemtime" attribute, we cannot implement a nice me

Re: Performance and Latency Chart for Flink

2016-09-16 Thread Timo Walther
and performance vs other competitors. Regards Sumit Chawla -- Freundliche Grüße / Kind Regards Timo Walther Follow me: @twalthr https://www.linkedin.com/in/twalthr

Re: DataStream of Future

2016-09-16 Thread Timo Walther
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus -- Freundliche Grüße / Kind Regards Timo Walther Follow me: @twalthr https://www.linkedin.com/in/twalthr

Re: [DISCUSS] Some thoughts about unify Stream SQL and Batch SQL grammer

2016-08-18 Thread Timo Walther
:) What do you think about this ? [1] https://calcite.apache.org/docs/stream.html - Jark Wu -- Freundliche Grüße / Kind Regards Timo Walther Follow me: @twalthr https://www.linkedin.com/in/twalthr

Re: [DISCUSS] Some thoughts about unify Stream SQL and Batch SQL grammer

2016-08-29 Thread Timo Walther
rg/jira/browse/CALCITE-1345 -- Freundliche Grüße / Kind Regards Timo Walther Follow me: @twalthr https://www.linkedin.com/in/twalthr

Re: [DISCUSS] Some thoughts about unify Stream SQL and Batch SQL grammer

2016-08-29 Thread Timo Walther
exception. Because we register the table as a regular table not streamable. - Jark Wu 在 2016年8月29日,下午8:13,Timo Walther <twal...@apache.org> 写道: Hi Jark, your code looks good and it also simplifies many parts. So the STREAM keyword is not optional but invalid now, right? What happens if

Re: RE:[DISCUSS] FLIP-11: Table API Stream Aggregations

2016-10-14 Thread Timo Walther
月13日 18:17 收件人: dev@flink.apache.org 抄送: Sean Wang; Timo Walther 主题: Re: 答复: RE:[DISCUSS] FLIP-11: Table API Stream Aggregations Hi Zhangrucong, yes, we want to use Calcite's SQL parser including its window syntax, i.e., - the standard SQL OVER windows (in streaming with a few r

Re: Move Row, RowInputFormat to core package

2016-11-25 Thread Timo Walther
Hi Anton, I would also support the idea of moving Row and RowTypeInfo to Flink core. I think there are many real-world use cases where a variable-length record that supports null values is required. However, I think that those classes needs to be reworked before. They should not depend on

Re: Type problem in RichFlatMapFunction when using GenericArray type

2016-10-11 Thread Timo Walther
as expected. Using Flink 1.1.2. Cheers, Martin [1] https://gist.github.com/s1ck/37aefb19198cd01a8b998fab354c2cfd -- Freundliche Grüße / Kind Regards Timo Walther Follow me: @twalthr https://www.linkedin.com/in/twalthr

Re: Type problem in RichFlatMapFunction when using GenericArray type

2016-10-11 Thread Timo Walther
I identified the problem and opened a issue for it: https://issues.apache.org/jira/browse/FLINK-4801 Am 11/10/16 um 15:31 schrieb Timo Walther: I will also have a look at this issue. Am 11/10/16 um 09:10 schrieb Chesnay Schepler: Yes, i think a JIRA issue would be good

Re: [FLINK-4604] Trouble with NULL type

2016-12-01 Thread Timo Walther
Hi Anton, I would open an issue and PR at Calcite. Until this issue is fixed, I would either copy or even better wrap the Calcite rule into a Flink specific rule that uses the correct representation of null. Regards, Timo Am 01/12/16 um 12:19 schrieb Anton Mushin: Hi everybody, In issue

Re: 1.2-SNAPSHOT documentation

2016-12-04 Thread Timo Walther
-- Freundliche Grüße / Kind Regards Timo Walther Follow me: @twalthr https://www.linkedin.com/in/twalthr

Re: [FLINK-4704 ] Move Table API to org.apache.flink.table

2016-12-05 Thread Timo Walther
Hi Anton, thanks for bringing up this discussion and volunteering. Yes we should do that before the 1.2 release. You can assign the issue to you if you like. However, we should first get most of the pull requests in (esp. the large PRs). The next release is still some weeks away so it make

Re: [FLINK-3615] Add support for non-native SQL types

2017-01-09 Thread Timo Walther
You are right. The issue has been solved. I closed it. Timo Am 09/01/17 um 11:58 schrieb Till Rohrmann: Hi Alex, I guess that Fabian and Timo (cc'ed) will be able to answer this question. Cheers, Till On Wed, Dec 28, 2016 at 9:38 AM, Alexander Chermenin

Re: [VOTE] Release Apache Flink 1.2.1 (RC1)

2017-03-29 Thread Timo Walther
A user reported that all tumbling and slinding window assigners contain a pretty obvious bug about offsets. https://issues.apache.org/jira/browse/FLINK-6214 I think we should also fix this for 1.2.1. What do you think? Regards, Timo Am 29/03/17 um 11:30 schrieb Robert Metzger: Hi Haohui, I

Re: TumblingEventTimeWindows with negative offset / Wrong documentation

2017-03-29 Thread Timo Walther
Hi Vladislav, thank you very much for reporting this. You are right this is bug. I opened an issue for it: https://issues.apache.org/jira/browse/FLINK-6214 I think it will be solved soon. Regards, Timo Am 29/03/17 um 16:57 schrieb Vladislav Pernin: Hi, The documentation mentions the

Re: Flink on yarn passing yarn config params.

2017-03-29 Thread Timo Walther
Hi, you can pass application tags using `yarn.tags` option. See also here for more options: https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/config.html#yarn I hope that helps. Regards, Timo Am 29/03/17 um 01:18 schrieb praveen kanamarlapudi: yarn application tag

Re: [DISCUSS] Project build time and possible restructuring

2017-03-20 Thread Timo Walther
I agress with Aljoscha that we might consider moving from Jenkins to Travis. Is there any disadvantage in using Jenkins? I think we should structure the project according to release management (e.g. more frequent releases of libraries) or other criteria (e.g. core and non-core) instead of

Re: FW: [DISCUSS] Table API / SQL indicators for event and processing time

2017-03-20 Thread Timo Walther
) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -Original Message- From: Timo Walther [mailto:twal...@apache.org] Sent: Monday, March 20, 2017 12:29 PM To: Radu Tudoran

Re: [DISCUSS] Table API / SQL indicators for event and processing time

2017-03-20 Thread Timo Walther
sum as cnt2) SQL: addTable[(Int, String, Long)]("MyTable", 'a, 'b, 'c) .registeredWatermarks('a, waterMarkFunction1) SELECT a, SUM(b) OVER (PARTITION BY c ORDER BY a RANGE BETWEEN 2 PRECEDING) AS sumB FROM MyTable What do you think ? 2017-02-22 23:44 GMT+08:00 Ti

Re: [DISCUSS] Table API / SQL indicators for event and processing time

2017-03-20 Thread Timo Walther
mean we can no longer work with count windows? There are a lot of queries where windows would be defined based on cardinality of elements. -Original Message- From: Timo Walther [mailto:twal...@apache.org] Sent: Monday, March 20, 2017 10:08 AM To: dev@flink.apache.org Subject: Re: [DISCUSS]

Re: [DISCUSS] Project build time and possible restructuring

2017-03-20 Thread Timo Walther
20, 2017 at 1:48 PM, Timo Walther <twal...@apache.org> wrote: I agress with Aljoscha that we might consider moving from Jenkins to Travis. Is there any disadvantage in using Jenkins? I think we should structure the project according to release management (e.g. more frequent releases of lib

Re: [DISCUSS] Project build time and possible restructuring

2017-03-21 Thread Timo Walther
So what do we want to move to the libraries repository? I would propose to move these modules first: flink-cep-scala flink-cep flink-gelly-examples flink-gelly-scala flink-gelly flink-ml All other modules (e.g. in flink-contrib) are rather connectors. I think it would be better to move those

Re: Dropping Java 7 support

2017-07-18 Thread Timo Walther
Hurray! Finally IntStreams, LongStreams, etc. in our stream processor ;-) Timo Am 18.07.17 um 16:31 schrieb Stephan Ewen: Hi all! Over the last days, there was a longer poll running concerning dropping the support for Java 7. The feedback from users was unanimous - in favor of dropping Java

Re: Using native library in Flink

2017-07-18 Thread Timo Walther
Hi Mike, do you run Flink locally or in a cluster? You have to make sure that VM argument -Djava.library.path is set for all Flink JVMs. Job Manager and Task Managers might run in separate JVMs. Make also sure that the library is accessible from all node. I don't know what happens if the

Re: [DISCUSS] Release 1.3.2 planning

2017-07-24 Thread Timo Walther
041 < https://issues.apache.org/jira/browse/FLINK-7041> I’m trying to quickly provide a fix. On 26. Jun 2017, at 15:30, Timo Walther <twal...@apache.org> wrote: I just opened a PR which should be included in the next bug fix release for the Table API: https://issues.apache.org/jir

Fwd: Re: AVRO Union type support in Flink

2017-07-19 Thread Timo Walther
Weitergeleitete Nachricht Betreff:Re: AVRO Union type support in Flink Datum: Wed, 19 Jul 2017 10:26:24 -0400 Von:Vishnu Viswanath <vishnu.viswanat...@gmail.com> An: Timo Walther <twal...@apache.org> Hi Timo, Thanks for checking that. I did not try yet

Re: How to run wordcount program on Multi-node Cluster.

2017-08-07 Thread Timo Walther
Hi Ramanji, you can find the source code of the examples here: https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/wordcount/WordCount.java A general introduction how the cluster execution works can be found here:

Re: How to run wordcount program on Multi-node Cluster.

2017-08-07 Thread Timo Walther
253) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702) at java.lang.Thread.run(Thread.java:745) On Mon, Aug 7, 2017 at 6:49 PM, Timo Walther <twal...@apache.org> wrote: Make sure that the file exists and is accessible from all Flink tasks managers. Am 07.08.17 um 14:35 schrieb P. Ramanjan

Re: How to run wordcount program on Multi-node Cluster.

2017-08-07 Thread Timo Walther
) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:95) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:263) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702) at java.lang.Thread.run(Thread.java:748) On Mon, Aug 7, 2017 at 5:56 PM, Timo Walther

Re: [DISCUSS] Stop serving docs for Flink version prior to 1.0

2017-08-23 Thread Timo Walther
I had a offline discussion with Ufuk. I looked into the docs build scripts recently, I can take care of removing the old docs. There other issues that need to be fixed before the next release as well: - Drop docs < 1.0 - Make Javadocs with Scala build again - Build all docs >= 1.0 again (esp.

Re: [DISCUSS] Flink 1.4 and time based release

2017-08-23 Thread Timo Walther
I also think we shouldn't publish releases regularly, just to have a release regularly. Maybe we can do time-based releases more flexible: Instead of feature-freeze after 3 months, 1 month testing. We could do it like feature-freeze 3 months after the last release, unlimited testing. This

Re: [DISCUSS] Release Apache Flink 1.3.1

2017-06-19 Thread Timo Walther
I'm working on https://issues.apache.org/jira/browse/FLINK-6896 and https://issues.apache.org/jira/browse/FLINK-6881. I try to open a PR for both today. Timo Am 19.06.17 um 14:54 schrieb Robert Metzger: Fabian and SunJincheng, it looks like we are cancelling the 1.3.1 RC1. So there is the

Re: [DISCUSS] Release Apache Flink 1.3.1

2017-06-20 Thread Timo Walther
FLINK-6881 and FLINK-6896 are merged. The Table API is ready for a new RC. Timo Am 19.06.17 um 17:00 schrieb jincheng sun: Thanks @Timo! 2017-06-19 22:02 GMT+08:00 Timo Walther <twal...@apache.org>: I'm working on https://issues.apache.org/jira/browse/FLINK-6896 and

Re: [DISCUSS] Release 1.3.2 planning

2017-06-26 Thread Timo Walther
I just opened a PR which should be included in the next bug fix release for the Table API: https://issues.apache.org/jira/browse/FLINK-7005 Timo Am 23.06.17 um 14:09 schrieb Robert Metzger: Thanks Haohui. The first main task for the release management is to come up with a timeline :) Lets

Re: [VOTE] Release Apache Flink 1.3.1 (RC2)

2017-06-22 Thread Timo Walther
+1 (binding) I tested the following: - built from source - tested the web interface - ran some streaming programs It seems that Flink cannot be built if the path contains spaces. I added an issue for this (https://issues.apache.org/jira/browse/FLINK-6987). It seems that this error was

Re: [DISCUSS] Release 1.3.0 RC1 (Non voting, testing release candidate)

2017-05-24 Thread Timo Walther
I tested the quickstarts, the SBT build, the PlanVisualizer, and the HistoryServer. I could not find any serious problems. However, we have to update the quickstart scripts, once 1.3 is released. Timo Am 24.05.17 um 16:05 schrieb Chesnay Schepler: I've found a small problem in the yarn

Re: [VOTE] Release Apache Flink 1.3.0 (RC3)

2017-05-31 Thread Timo Walther
I don't think that FLINK-6780 is a blocker, because the Table API is still a new feature. FLINK-6736 was also a hard bug. However, if there will be a RC4, a fix should be included. Regards, Timo Am 31.05.17 um 02:55 schrieb Haohui Mai: Hi, We have discovered

Re: [VOTE] Release Apache Flink 1.3.0 (RC3)

2017-05-31 Thread Timo Walther
We should also include FLINK-6783. It seems that WindowedStream::aggregate is broken right now. Am 31.05.17 um 14:31 schrieb Timo Walther: I merged all Table API related PRs. I'm also fine with a 1.3.1 release this or next week. Am 31.05.17 um 14:08 schrieb Till Rohrmann: I would be ok

Re: [VOTE] Release Apache Flink 1.3.0 (RC3)

2017-05-31 Thread Timo Walther
Walther <twal...@apache.org> wrote: We should also include FLINK-6783. It seems that WindowedStream::aggregate is broken right now. Am 31.05.17 um 14:31 schrieb Timo Walther: I merged all Table API related PRs. I'm also fine with a 1.3.1 release this or next week. Am 31.05.17 um 14:08 s

Re: [VOTE] Release Apache Flink 1.3.0 (RC3)

2017-05-31 Thread Timo Walther
I merged all Table API related PRs. I'm also fine with a 1.3.1 release this or next week. Am 31.05.17 um 14:08 schrieb Till Rohrmann: I would be ok to quickly release 1.3.1 once the the respective PRs have been merged. Just for your information, I'm not yet through with the testing of the

Re: flink-table sql overlaps time.scala

2017-05-31 Thread Timo Walther
Hi Stefano, I implemented the overlap according to Calcite's implementation. Maybe they changed the behavior in the mean time. I agree we should try to stay in sync with Calcite. What do other DB vendors do? Feel free to open an issue about this. Regards, Timo Am 30.05.17 um 14:24 schrieb

Fwd: Re: Flink Watermark and timing

2017-10-04 Thread Timo Walther
Nachricht Betreff:Re: Flink Watermark and timing Datum: Tue, 3 Oct 2017 06:37:13 +0200 Von:Björn Zachrisson <bjo...@gmail.com> An: Timo Walther <twal...@apache.org> Hi Timo, One more question regarding that to clarify. Where do i specify in which window a event tha

Re: Metrics to data connectors

2017-10-02 Thread Timo Walther
@Chesnay do you know more about it? Am 10/1/17 um 4:45 PM schrieb Michael Fong: Hi, all, Is there any existing metrics implemented to any src/sink connectors? I am thinking to add some metrics to C* connector that might help to give a early sign of scalability problem (like back pressure)

Re: partial upgrade

2017-10-02 Thread Timo Walther
Hi Chen, I think in a long-term perspective it makes sense to support things like this. The next big step is dynamic scaling without stopping the execution. Partial upgrades could be addressed afterwards, but I'm not aware of any plans. Until then, I would recommend a different architecture

[DISCUSS] FLIP-24 - SQL Client

2017-12-19 Thread Timo Walther
Hey everyone, in the past, the community already asked about having a way to write Flink jobs without extensive programming skills. During the last year we have put a lot of effort in the core of our Table & SQL API. Now it is time to improve the tooling around it as well and make Flink more

Re: [VOTE] Release 1.4.0, release candidate #3

2017-12-11 Thread Timo Walther
+1 (binding) - build the source locally - run various table programs - checked the resource consumption of table programs with retention enabled and disabled - built a quickstart project - tested the web ui submission (found https://issues.apache.org/jira/browse/FLINK-8187 but this is

Re: Using SQL with dynamic tables where rows are updated

2017-12-20 Thread Timo Walther
Hi Ghassan, in your example the result 3.5 is correct. The query is executed with standard SQL semantics. You only group by ProductID and since it is the same for all elements, the average is 3.5. The second "review-3" does not replace anything. In general, the replacement would happen in

Re: Support more intelligent function lookup in FunctionCatalog for UDF

2018-05-14 Thread Timo Walther
Hi Rong, yes I think we can improve the type infererence at this point. Input parameter type inference can be more tolerant but return types should be as exact as possible. The change should only touch ScalarSqlFunction and UserDefinedFunctionUtils#createEvalOperandTypeInference, right?

Re: Support more intelligent function lookup in FunctionCatalog for UDF

2018-05-15 Thread Timo Walther
ment/d/1zKSY1z0lvtQdfOgwcLnCMSRHew3weeJ6QfQjSD0zWas/edit?usp=sharing On Mon, May 14, 2018 at 4:36 AM, Timo Walther <twal...@apache.org> wrote: Hi Rong, yes I think we can improve the type infererence at this point. Input parameter type inference can be more tolerant but return types should be as exact as possible. The change

  1   2   3   4   5   6   7   >