Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

2016-10-05 Thread Vishnu Viswanath
Thank you Aljoscha, Yes, I agree we don't need ProcessingTimeEvcitor. I will change the current TimeEvictors to use EventTimeEvictor as suggested. Also, figure out a way to pass timestamp to Evictor interface so that we can avoid exposing StreamRecrods. Regards, Vishnu On Tue, Sep 20, 2016 at

Re: Duplicate sort keys

2016-10-05 Thread Fabian Hueske
Hi Greg, IMO you are right. We should remove duplicate sort keys. Best, Fabian 2016-10-03 16:04 GMT+02:00 Greg Hogan : > Is it correct to expect that Flink should remove duplicate sort keys? I'm > working on instrumenting the FixedLengthRecordSorter (FLINK-4705) and the > following test case fr

Re: Releasing Flink 1.1.3

2016-10-05 Thread Ufuk Celebi
Thanks for your effort! I'm going to kick off the build now. :-) On Wed, Oct 5, 2016 at 5:59 PM, Maximilian Michels wrote: > Kostas PR https://github.com/apache/flink/pull/2593 is merged. I think > we're good to go. > > On Wed, Oct 5, 2016 at 3:44 PM, Maximilian Michels wrote: >> I'll merge htt

[jira] [Created] (FLINK-4751) Extend Flink's futures to support combining two futures

2016-10-05 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4751: Summary: Extend Flink's futures to support combining two futures Key: FLINK-4751 URL: https://issues.apache.org/jira/browse/FLINK-4751 Project: Flink Issue T

Re: Releasing Flink 1.1.3

2016-10-05 Thread Maximilian Michels
Kostas PR https://github.com/apache/flink/pull/2593 is merged. I think we're good to go. On Wed, Oct 5, 2016 at 3:44 PM, Maximilian Michels wrote: > I'll merge https://github.com/apache/flink/pull/2548 for the release. > It's cosmetic but it avoids a NPE in case the user jar doesn't contain > Fli

Re: KafkaProducer can not be instantiated

2016-10-05 Thread Tzu-Li (Gordon) Tai
Sorry, correction to my last statements: On the consumer side I think the instantiation was already removed from the constructor in a recent commit. On October 5, 2016 at 11:37:41 PM, Tzu-Li (Gordon) Tai (tzuli...@apache.org) wrote: This matters on the consumer side, yes. Moving the instantiat

Re: KafkaProducer can not be instantiated

2016-10-05 Thread Tzu-Li (Gordon) Tai
This matters on the consumer side, yes. Moving the instantiation out of the constructor will require such guarantee that the list fetched individually at subtasks are determinate and identical. On the producer side I don’t really think it matters. Unless the user implementations of the provided

Re: KafkaProducer can not be instantiated

2016-10-05 Thread Chesnay Schepler
if you were to move the partition list fetching out of the constructor int open(), is there any guarantee that for each fetching subtask the partition list is identical? On 05.10.2016 17:17, Tzu-Li (Gordon) Tai wrote: Hi Timo, I haven’t had the chance to look at the producer side too much yet

Re: KafkaProducer can not be instantiated

2016-10-05 Thread Tzu-Li (Gordon) Tai
Hi Timo, I haven’t had the chance to look at the producer side too much yet, but after a look in the code, I think it’s reasonable to remove the instantiation from the producer constructor. The instantiation in the constructor is only used for partition list fetching & eager properties validati

KafkaProducer can not be instantiated

2016-10-05 Thread Timo Walther
Hey everyone, I'm currently rewriting the KafkaTabeSinkTest and discovered something that doesn't seem to be intended: Is it intended that FlinkKafkaProducer08 cannot be instantiated without a running Kafka instance? The constructor of FlinkKafkaProducerBase calls getKafkaProducer() which a

[jira] [Created] (FLINK-4750) Ensure that active processing time triggers complete before closing operators

2016-10-05 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-4750: --- Summary: Ensure that active processing time triggers complete before closing operators Key: FLINK-4750 URL: https://issues.apache.org/jira/browse/FLINK-4750 Project: Fl

[jira] [Created] (FLINK-4749) Remove redundant processing time timers and futures in the window operator

2016-10-05 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-4749: --- Summary: Remove redundant processing time timers and futures in the window operator Key: FLINK-4749 URL: https://issues.apache.org/jira/browse/FLINK-4749 Project: Flink

[jira] [Created] (FLINK-4748) Fix shutdown of automatic watermark context

2016-10-05 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-4748: --- Summary: Fix shutdown of automatic watermark context Key: FLINK-4748 URL: https://issues.apache.org/jira/browse/FLINK-4748 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-4747) Instability due to pending processing timers on operator close

2016-10-05 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-4747: --- Summary: Instability due to pending processing timers on operator close Key: FLINK-4747 URL: https://issues.apache.org/jira/browse/FLINK-4747 Project: Flink I

Re: Releasing Flink 1.1.3

2016-10-05 Thread Maximilian Michels
I'll merge https://github.com/apache/flink/pull/2548 for the release. It's cosmetic but it avoids a NPE in case the user jar doesn't contain Flink jobs. On Wed, Oct 5, 2016 at 12:59 PM, Kostas Kloudas wrote: > Hi Ufuk, > > Thanks for being the release manager. > > There is already an open PR unde

[jira] [Created] (FLINK-4745) Convert KafkaTableSource test to unit tests

2016-10-05 Thread Timo Walther (JIRA)
Timo Walther created FLINK-4745: --- Summary: Convert KafkaTableSource test to unit tests Key: FLINK-4745 URL: https://issues.apache.org/jira/browse/FLINK-4745 Project: Flink Issue Type: Improveme

[jira] [Created] (FLINK-4746) Replace TaskManagerRuntimeInfo by TaskManagerConfiguration

2016-10-05 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-4746: Summary: Replace TaskManagerRuntimeInfo by TaskManagerConfiguration Key: FLINK-4746 URL: https://issues.apache.org/jira/browse/FLINK-4746 Project: Flink Issu

[jira] [Created] (FLINK-4744) Introduce usercode class loader to deserialize partitionable operator state

2016-10-05 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-4744: - Summary: Introduce usercode class loader to deserialize partitionable operator state Key: FLINK-4744 URL: https://issues.apache.org/jira/browse/FLINK-4744 Project:

Re: [Discuss] Upgrade JUnit to 4.12

2016-10-05 Thread Greg Hogan
Tests are passing with one additional change to an inner test class visibility. The ticket is FLINK-4740. On Wed, Oct 5, 2016 at 3:52 AM, Till Rohrmann wrote: > +1 for that :-) > > On Tue, Oct 4, 2016 at 10:11 PM, Greg Hogan wrote: > > > PowerMock reports "org.powermock.reflect.exceptions. > >

Re: Releasing Flink 1.1.3

2016-10-05 Thread Kostas Kloudas
Hi Ufuk, Thanks for being the release manager. There is already an open PR under review for the Continuous File Sources. Kostas > On Oct 4, 2016, at 11:25 PM, Ufuk Celebi wrote: > > Thanks for your prompt responses. > > Except for the streaming file source issues, all mentioned issues are >

RE: Some questions about Table API and FlinkSQL

2016-10-05 Thread Anton Mushin
Hi Timo, I didn't find the similar issue in Jira and created a new one: FLINK-4743 Thanks for help! Best regards, Anton Mushin -Original Message- From: Timo Walther [mailto:twal...@apache.org] Sent: Tuesday, October 04, 2016 8:44 PM To: dev@flink.apache.org Subject: Re: Some questions ab

[jira] [Created] (FLINK-4743) The sqrt/power function not accept the real data types.

2016-10-05 Thread Anton Mushin (JIRA)
Anton Mushin created FLINK-4743: --- Summary: The sqrt/power function not accept the real data types. Key: FLINK-4743 URL: https://issues.apache.org/jira/browse/FLINK-4743 Project: Flink Issue Typ

Re: [Discuss] Upgrade JUnit to 4.12

2016-10-05 Thread Till Rohrmann
+1 for that :-) On Tue, Oct 4, 2016 at 10:11 PM, Greg Hogan wrote: > PowerMock reports "org.powermock.reflect.exceptions. > FieldNotFoundException: > Field 'fTestClass' was not found in class > org.junit.internal.runners.MethodValidator." > https://github.com/jayway/powermock/issues/551 > > Th