No data issued by flink window after a few hours

2018-10-09 Thread ? ??
Hi all, I used flink window, and when the job begins, we could get the results of windiow.But there’re no results issued after a few hours. I found the job is still running and no errors, and the data not used window all can be issued. By the way, I used Flink 1.3.2 and ram to cache

[jira] [Created] (FLINK-10520) Job save points REST API fails unless parameters are specified

2018-10-09 Thread Elias Levy (JIRA)
Elias Levy created FLINK-10520: -- Summary: Job save points REST API fails unless parameters are specified Key: FLINK-10520 URL: https://issues.apache.org/jira/browse/FLINK-10520 Project: Flink

[jira] [Created] (FLINK-10519) flink-parent:1.6.1 artifact can't be found on maven central

2018-10-09 Thread Florian Schmidt (JIRA)
Florian Schmidt created FLINK-10519: --- Summary: flink-parent:1.6.1 artifact can't be found on maven central Key: FLINK-10519 URL: https://issues.apache.org/jira/browse/FLINK-10519 Project: Flink

Re: Sharing state between subtasks

2018-10-09 Thread Elias Levy
On Tue, Oct 9, 2018 at 1:25 AM Aljoscha Krettek wrote: > @Elias Do you know if Kafka Consumers do this alignment across multiple > consumers or only within one Consumer across the partitions that it reads > from. > The behavior is part of Kafka Streams

[DISCUSS] Integrate Flink SQL well with Hive ecosystem

2018-10-09 Thread Zhang, Xuefu
Hi all, Along with the community's effort, inside Alibaba we have explored Flink's potential as an execution engine not just for stream processing but also for batch processing. We are encouraged by our findings and have initiated our effort to make Flink's SQL capabilities full-fledged. When

Re: [DISCUSS] Flink Cluster Overview Dashboard Improvement Proposal

2018-10-09 Thread Jin Sun
Great job! That would very helpful for debug. I would suggest to use small icons for this Job Manager/Managers when there are too many instances (like a thousand) May be we can also introduce locality, that task managers belongs to same rack shows together? Small icons can be like this:

Re: [DISCUSS] [Contributing] (2) - Review Steps

2018-10-09 Thread Hequn Cheng
+1 On Tue, Oct 9, 2018 at 3:25 PM Till Rohrmann wrote: > +1 > > On Tue, Oct 9, 2018 at 9:08 AM Zhijiang(wangzhijiang999) > wrote: > > > +1 > > -- > > 发件人:vino yang > > 发送时间:2018年10月9日(星期二) 14:08 > > 收件人:dev > > 主 题:Re: [DISCUSS]

[ANNOUNCE] Weekly community update #41

2018-10-09 Thread Till Rohrmann
Dear community, this is the weekly community update thread #41. Please post any news and updates you want to share with the community to this thread. # Feature freeze for Flink 1.7 The community has decided to freeze the feature development for Flink 1.7.0 on the 22nd of October [1]. # Flink

Re: [DISCUSS] Flink Cluster Overview Dashboard Improvement Proposal

2018-10-09 Thread Till Rohrmann
Hi Fabian, thanks for starting this discussion. I agree with you that Flink's web dashboard lacks a bit of general cluster overview information on the front page. Your mock looks really promising to me since it shows some basic metrics and cluster information at a glance. Apart from the the

[jira] [Created] (FLINK-10518) Inefficient design in ContinuousFileMonitoringFunction

2018-10-09 Thread Huyen Levan (JIRA)
Huyen Levan created FLINK-10518: --- Summary: Inefficient design in ContinuousFileMonitoringFunction Key: FLINK-10518 URL: https://issues.apache.org/jira/browse/FLINK-10518 Project: Flink Issue

Re: [DISCUSS] Flink Cluster Overview Dashboard Improvement Proposal

2018-10-09 Thread Fabian Wollert
argh, i think the screenshot is missing (at least nabble is not showing anything). here is a link to the mockup: https://drive.google.com/file/d/1p3wVP028_AFFLZ6fjPb41yAI8zUhgDTO/view?usp=sharing Cheers -- *Fabian WollertZalando SE* E-Mail: fab...@zalando.de Am Di., 9. Okt. 2018 um 12:46

[DISCUSS] Flink Cluster Overview Dashboard Improvement Proposal

2018-10-09 Thread Fabian Wollert
Hi everyone, disclaimer: i read the contribution guide about improvement requests (i.e. i should actually just start a jira ticket) but i thought it would make sense to run this first through the mailing list here. after collecting some input i would then create the jira ticket. When accessing

[jira] [Created] (FLINK-10517) Add stability test for the REST API

2018-10-09 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-10517: Summary: Add stability test for the REST API Key: FLINK-10517 URL: https://issues.apache.org/jira/browse/FLINK-10517 Project: Flink Issue Type:

Re: Sharing state between subtasks

2018-10-09 Thread Fabian Hueske
Hi, I think watermark / event-time skew is a problem that many users are struggling with. A built-in primitive to align event-time would be a great feature! However, there are also some cases when it would be useful for different streams to have diverging event-time, such as an interval join [1]

Re: Sharing state between subtasks

2018-10-09 Thread Aljoscha Krettek
Yes, I think this is the way to go. This would also go well with a redesign of the source interface that has been floated for a while now. I also created a prototype a while back: https://github.com/aljoscha/flink/tree/refactor-source-interface

Re: [DISCUSS] Dropping flink-storm?

2018-10-09 Thread Fabian Hueske
Yes, let's do it this way. The wrapper classes are probably not too complex and can be easily tested. We have the same for the Hadoop interfaces, although I think only the Input- and OutputFormatWrappers are actually used. Am Di., 9. Okt. 2018 um 09:46 Uhr schrieb Chesnay Schepler <

Re: [DISCUSS] Dropping flink-storm?

2018-10-09 Thread Chesnay Schepler
That sounds very good to me. On 08.10.2018 11:36, Till Rohrmann wrote: Good point. The initial idea of this thread was to remove the storm compatibility layer completely. During the discussion I realized that it might be useful for our users to not completely remove it in one go. Instead for

Re: [DISCUSS] [Contributing] (2) - Review Steps

2018-10-09 Thread Till Rohrmann
+1 On Tue, Oct 9, 2018 at 9:08 AM Zhijiang(wangzhijiang999) wrote: > +1 > -- > 发件人:vino yang > 发送时间:2018年10月9日(星期二) 14:08 > 收件人:dev > 主 题:Re: [DISCUSS] [Contributing] (2) - Review Steps > > +1 > > Peter Huang 于2018年10月9日周二

[jira] [Created] (FLINK-10516) YarnApplicationMasterRunner fail to initialize FileSystem with correct Flink Configuration during setup

2018-10-09 Thread Shuyi Chen (JIRA)
Shuyi Chen created FLINK-10516: -- Summary: YarnApplicationMasterRunner fail to initialize FileSystem with correct Flink Configuration during setup Key: FLINK-10516 URL:

Re: Handling burst I/O when using tumbling/sliding windows

2018-10-09 Thread Piotr Nowojski
Hi, Sorry for getting back so late and thanks for the improved document :) I think now I got your idea. You are now trying (or have you already done it?) to implement a custom window assigner, that would work as in the [Figure 3] from your document? I think that indeed should be possible

回复:[DISCUSS] [Contributing] (2) - Review Steps

2018-10-09 Thread Zhijiang(wangzhijiang999)
+1 -- 发件人:vino yang 发送时间:2018年10月9日(星期二) 14:08 收件人:dev 主 题:Re: [DISCUSS] [Contributing] (2) - Review Steps +1 Peter Huang 于2018年10月9日周二 下午1:54写道: > +1 > > On Mon, Oct 8, 2018 at 7:47 PM Thomas Weise wrote: > > > +1 > > > > > >

Re: [DISCUSS] [Contributing] (2) - Review Steps

2018-10-09 Thread vino yang
+1 Peter Huang 于2018年10月9日周二 下午1:54写道: > +1 > > On Mon, Oct 8, 2018 at 7:47 PM Thomas Weise wrote: > > > +1 > > > > > > On Mon, Oct 8, 2018 at 7:36 PM Tzu-Li Chen wrote: > > > > > +1 > > > > > > Jin Sun 于2018年10月9日周二 上午2:10写道: > > > > > > > +1, look forward to see the change. > > > > > > > >