Re: [DISCUSS][TABLE] How to handle empty delete for UpsertSource

Hi Piotrek, Great to see your replies, and really thanks for all your suggestions. Inline is a good way, i will do it same as you :-) *> I’m assuming that we are talking about event time and that `(delete 1, a, 1)` happened before `(add 1, a, 2)`, right?* We are talking about processing

[jira] [Created] (FLINK-10205) Batch Job: InputSplit Fault tolerant for DataSourceTask

JIN SUN created FLINK-10205: --- Summary: Batch Job: InputSplit Fault tolerant for DataSourceTask Key: FLINK-10205 URL: https://issues.apache.org/jira/browse/FLINK-10205 Project: Flink Issue Type:

[jira] [Created] (FLINK-10204) Job is marked as FAILED after serialization exception

Ben La Monica created FLINK-10204: - Summary: Job is marked as FAILED after serialization exception Key: FLINK-10204 URL: https://issues.apache.org/jira/browse/FLINK-10204 Project: Flink

Re: Flink 1.7 Development Priorities

Any plans to advance the "splittable source" idea? It probably won't be ready to implement, but important to get on the radar. Thanks, Thomas On Thu, Aug 23, 2018 at 1:12 AM Aljoscha Krettek wrote: > Hi Everyone, > > After the recent Flink 1.6 release the people working on Flink at data >

Re: Flink 1.7 Development Priorities

I would be nice to see the state TTL work finished. At the moment state will not expire if a key is not accessed while the job is running, which means jobs that require state expiration must continue using timers to expire data. On Thu, Aug 23, 2018 at 1:12 AM Aljoscha Krettek wrote: > Hi

Re: Flink cluster crashing going from 1.4.0 -> 1.5.3

parallelism is 100. I tried clusters with 1 and 2 slots per TM yielding 100 or 50 TMs in cluster. I did notice that URL http://jobmanager:port/jobs/job_id/metrics in 1.5.x returns huge list of "latency.source_id. " IDs. Heap dump shows that hash map takes 1.6GB for me. I am guessing that

Re: Flink cluster crashing going from 1.4.0 -> 1.5.3

Hi, How many task slots do you have in the cluster and per machine, and what parallelism are you using? Piotrek > On 23 Aug 2018, at 16:21, Jozef Vilcek wrote: > > Yes, on smaller data and therefore smaller resources and parallelism > exactly same job runs fine > > On Thu, Aug 23, 2018,

Re: Support Hadoop 2.6 for StreamingFileSink

Hi guys, I've created a ticket for that issue in Jira and proposed possible solution just to continue our discussion and develop a plan how to fix the issue. https://issues.apache.org/jira/browse/FLINK-10203 Cheers, Artsem On Tue, 21 Aug 2018 at 16:59, Artsem Semianenka wrote: > Thanks

Re: Flink cluster crashing going from 1.4.0 -> 1.5.3

Yes, on smaller data and therefore smaller resources and parallelism exactly same job runs fine On Thu, Aug 23, 2018, 16:11 Aljoscha Krettek wrote: > Hi, > > So with Flink 1.5.3 but a smaller parallelism the job works fine? > > Best, > Aljoscha > > > On 23. Aug 2018, at 15:25, Jozef Vilcek

Re: Flink cluster crashing going from 1.4.0 -> 1.5.3

Hi, So with Flink 1.5.3 but a smaller parallelism the job works fine? Best, Aljoscha > On 23. Aug 2018, at 15:25, Jozef Vilcek wrote: > > Hello, > > I am trying to get my Beam application (run on newer version of Flink > (1.5.3) but having trouble with that. When I submit application,

Flink cluster crashing going from 1.4.0 -> 1.5.3

Hello, I am trying to get my Beam application (run on newer version of Flink (1.5.3) but having trouble with that. When I submit application, everything works fine but after a few mins (as soon as 2 minutes after job start) cluster just goes bad. Logs are full of timeouts for heartbeats,

[jira] [Created] (FLINK-10203) HadoopRecoverableWriter does not support Hadoop

Artsem Semianenka created FLINK-10203: - Summary: HadoopRecoverableWriter does not support Hadoop Key: FLINK-10203 URL: https://issues.apache.org/jira/browse/FLINK-10203 Project: Flink

Re: Flink 1.7 Development Priorities

Hi, I think Kafka 1.0/1.1/2.0 connector can be part of the Flink 1.7 release. The current kafka 1.0 connector PR has been submitted. I am refactoring the existing kafka connector test code to reduce the amount of duplicate code. Thanks, vino. Coo1 min 于2018年8月23日周四 下午5:59写道： > Hi, > > I am

Allowed lateness for coGroup/join operators

Hi folks I'm surprised coGroup and join doesn't support allowedLateness feature (while underlying WindowedStream does) in the current release (1.5/1.6). I've opened a jira ticket with more details and ready to make appropriate changes in API: https://issues.apache.org/jira/browse/FLINK-10050 I'm

Re: [DISCUSS][TABLE] How to handle empty delete for UpsertSource

Hi, Thanks for very detailed explanation :) Please check my inlined responses. > On 22 Aug 2018, at 14:28, Hequn Cheng wrote: > > Hi all, > Thanks a lot for your replies, @Piotr Nowojski > @Fabian Hueske > @Xingcan Cui

Re: Flink 1.7 Development Priorities

Hi, I am concerned about the progress of CEP library development, Can the following two main feature be kicked off and be involed in Flink1.7? 1) integration of CEP & SQL 2) dynamic change of CEP pattern without the downtime And i am willing to contribute to this, thx. Aljoscha Krettek

Re: Flink 1.7 Development Priorities

Hi: Can we include task manager isolation for jobs in 1.7.0? This is a feature that blocks us from deploying flink more widely in our compan? I've opened issues for that and want to help to contribute that, but lacks guidance and reviews from

[jira] [Created] (FLINK-10202) Enable configuration for state.checkpoint.dir with environment

buptljy created FLINK-10202: --- Summary: Enable configuration for state.checkpoint.dir with environment Key: FLINK-10202 URL: https://issues.apache.org/jira/browse/FLINK-10202 Project: Flink Issue

Flink 1.7 Development Priorities

Hi Everyone, After the recent Flink 1.6 release the people working on Flink at data Artisans came together to talk about what we want to work on for Flink 1.7. The following is a list of high-level directions that we will be working on for the next couple of months. This doesn't mean that