[jira] [Created] (FLINK-4419) Batch improvement for supporting dfs as a ResultPartitionType

shuai.xu created FLINK-4419: --- Summary: Batch improvement for supporting dfs as a ResultPartitionType Key: FLINK-4419 URL: https://issues.apache.org/jira/browse/FLINK-4419 Project: Flink Issue

Re: [DISCUSS] Move Redis and Flume connectors to Apache Bahir and redirect contributions there

Good start point. +1 - Jark Wu > 在 2016年8月17日，下午11:24，Robert Metzger 写道： > > Hi, > > following the discussions in [1], the community decided to offer > contributors to submit streaming connectors the Flink community can not > maintain to Apache Bahir (there is still

Re: [DISCUSS] FLIP-9: Trigger DSL

Thanks Aljioscha In fact the onFire() method proposed in the JIRA is also included in the FLIP. The onCleanup() I agree that it would be a nice addition as it makes the API more complete. Now we have an onMerge(), an onFire() and onCleanup() which allow a trigger to react to every milestone in

[jira] [Created] (FLINK-4418) ClusterClient/ConnectionUtils#findConnectingAddress fails immediately if InetAddress.getLocalHost throws exception

Shannon Carey created FLINK-4418: Summary: ClusterClient/ConnectionUtils#findConnectingAddress fails immediately if InetAddress.getLocalHost throws exception Key: FLINK-4418 URL:

Reducing the JIRA PR message verbosity

Hello, I have noticed there is a high verbosity coming from the PR into the JIRA which makes it hard to focus on the message or the content of the PR through the JIRA itself. Has there been a discussion over minimizing the details displayed back on the PR's JIRA? Maybe just add the PR link and

[jira] [Created] (FLINK-4417) Checkpoints should be subsumed by CheckpointID not, by timestamp

Stephan Ewen created FLINK-4417: --- Summary: Checkpoints should be subsumed by CheckpointID not, by timestamp Key: FLINK-4417 URL: https://issues.apache.org/jira/browse/FLINK-4417 Project: Flink

Re: Jenkins and Travis CI failing

Hi Mx, Thanks for the info. Thanks, Sunny T On Wed, Aug 17, 2016 at 8:21 AM Maximilian Michels < mailto:Maximilian Michels > wrote: a, pre, code, a:link, body { word-wrap: break-word !important; } Hi Sunny, We are just getting started with Jenkins and may have to fine

Re: Some thoughts about the lower-level Flink APIs

I agree with Vasia that for data scientist it's likely easier to learn the high-level api. I like the material from http://dataartisans.github.io/flink-training/ but all of them focus on the high level api. Maybe we could have a guide (blog post, lecture, whatever) on how to get into Flink as a

[jira] [Created] (FLINK-4416) Provide a programmatic way to load a Yarn config

Maximilian Michels created FLINK-4416: - Summary: Provide a programmatic way to load a Yarn config Key: FLINK-4416 URL: https://issues.apache.org/jira/browse/FLINK-4416 Project: Flink

Re: [DISCUSS] Move Redis and Flume connectors to Apache Bahir and redirect contributions there

+1 On Wed, Aug 17, 2016 at 5:24 PM, Robert Metzger wrote: > Hi, > > following the discussions in [1], the community decided to offer > contributors to submit streaming connectors the Flink community can not > maintain to Apache Bahir (there is still the option that we are

Re: [DISCUSS] FLIP-9: Trigger DSL

Hi, I opened this Jira which should help in implementing the Trigger DSL but is also independent in that it just enhances the range of things that can be done with a Trigger: https://issues.apache.org/jira/browse/FLINK-4415 Cheers, Aljoscha On Wed, 17 Aug 2016 at 14:38 Jark Wu

[jira] [Created] (FLINK-4415) Enhance Trigger Interface with onFire() and onCleanup()

Aljoscha Krettek created FLINK-4415: --- Summary: Enhance Trigger Interface with onFire() and onCleanup() Key: FLINK-4415 URL: https://issues.apache.org/jira/browse/FLINK-4415 Project: Flink

Re: [DISCUSS] Streaming connector contributions

Okay, I agree that this is something we should make very clear. I'll start a DISCUSS thread. On Wed, Aug 17, 2016 at 10:07 AM, Stephan Ewen wrote: > Cool to see that the Bahir people are so open and helpful! > > Concerning moving the connectors: I think we should set up a

[DISCUSS] Move Redis and Flume connectors to Apache Bahir and redirect contributions there

Hi, following the discussions in [1], the community decided to offer contributors to submit streaming connectors the Flink community can not maintain to Apache Bahir (there is still the option that we are moving popular connectors to the main Flink codebase) The ActiveMQ connector pull request

Re: Jenkins and Travis CI failing

Hi Sunny, We are just getting started with Jenkins and may have to fine tune the CI setup a bit. Apart from a few unreliable Flink tests, Travis has its own issues because resources tend to be heavily limited there. Feel free to contact one of the shepherds for the component:

[jira] [Created] (FLINK-4414) Remove restriction on RpcService.getAddress

Wenlong Lyu created FLINK-4414: -- Summary: Remove restriction on RpcService.getAddress Key: FLINK-4414 URL: https://issues.apache.org/jira/browse/FLINK-4414 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] FLIP-9: Trigger DSL

Hi Aljoscha, Kostas, thanks for your detailed explanation. It makes sense. According to the discarding and accumulating, the FLIP says “the mode of parent trigger overwrites that of its children”. That means Trigger decide whether to discard window contents after firing , right ? But I find

[jira] [Created] (FLINK-4413) Improve savepoint restore error messages

Gyula Fora created FLINK-4413: - Summary: Improve savepoint restore error messages Key: FLINK-4413 URL: https://issues.apache.org/jira/browse/FLINK-4413 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-4412) [py] Chaining does not properly handle broadcast variables

Chesnay Schepler created FLINK-4412: --- Summary: [py] Chaining does not properly handle broadcast variables Key: FLINK-4412 URL: https://issues.apache.org/jira/browse/FLINK-4412 Project: Flink

[jira] [Created] (FLINK-4411) [py] Chained dual input children are not properly propagated

Chesnay Schepler created FLINK-4411: --- Summary: [py] Chained dual input children are not properly propagated Key: FLINK-4411 URL: https://issues.apache.org/jira/browse/FLINK-4411 Project: Flink

Re: [DISCUSS] FLIP-9: Trigger DSL

Hi, I think that would blow up state since there can be several triggers that need this kind of state, Any and All come to mind, possibly. If each of those keeps state that's at least a byte per trigger. If the finished state were kept centrally by the TriggerRunner it would just be one byte for

Re: [DISCUSS] FLIP-9: Trigger DSL

Hi Aljoscha, On the Repeat.? addition, I think that each trigger will have to have its own implementation, e.g. the CountTrigger should just set a dummy value in the counter in order to know if it should fire again or not. In other case, we will have to add more state and this can lead to

Re: [DISCUSS] FLIP-9: Trigger DSL

Regarding Repeat.forever() and the default being to not repeat. The simple reason is that Beam (née Google Dataflow) provides basically the same thing with their trigger DSL and that their triggers behave like this. I think it would not be beneficial to have the same feature in two systems in that

Re: [DISCUSS] FLIP-9: Trigger DSL

Kostas already nicely explained this! I just want to give some theoretical background. I see the underlying idea of triggers similar to predicates, i.e. "EventTimeTrigger.afterEndOfWindow().withEarlyTrigger(earlyFiringTrigger)" translates to a predicate "(E and ET) or WT" (where E is a predicate

[jira] [Created] (FLINK-4410) Split checkpoint times into synchronous and asynchronous part

Ufuk Celebi created FLINK-4410: -- Summary: Split checkpoint times into synchronous and asynchronous part Key: FLINK-4410 URL: https://issues.apache.org/jira/browse/FLINK-4410 Project: Flink

[jira] [Created] (FLINK-4409) class conflict between jsr305-1.3.9.jar and flink-shaded-hadoop2-1.1.1.jar

Renkai Ge created FLINK-4409: Summary: class conflict between jsr305-1.3.9.jar and flink-shaded-hadoop2-1.1.1.jar Key: FLINK-4409 URL: https://issues.apache.org/jira/browse/FLINK-4409 Project: Flink

Re: [HEADS-UP] Dev Week 33

I've seen this in BEAM as well and I really like it. - I'm finishing up the initial work for FLIP-3 (Docs restructuring) and hope to post something to the mailing list later today about it. - I will start working on top of Aljoscha's and Stefan's pull request #2376 with an initial version for

Re: [DISCUSS] FLIP-9: Trigger DSL

Hello Jark Wu, Both of them will work in the new DSL. The idea is that there should be no restrictions on the combinations one can do. Coming to what does the early and the late trigger do, the early trigger will be responsible for specifying when the trigger should fire in the period between

Re: [HEADS-UP] Dev Week 33

Good initiative @Aljoscha. - Just a convenience link for FLIP-6: The feature branch can be found here https://github.com/apache/flink/tree/flip-6. - My main task is the review of the Mesos PR of Eron ( https://github.com/apache/flink/pull/2315) so that we can continue making progress there. On

Re: [DISCUSS] Streaming connector contributions

Cool to see that the Bahir people are so open and helpful! Concerning moving the connectors: I think we should set up a vote thread, or at least a discussion with the possibility to object. Removing someones code from the repository is a bit of a sensitive thing in my experience, so let's make

Re: [DISCUSS] Streaming connector contributions

Bahir is now creating a second repository "bahir-flink" for our connectors: https://issues.apache.org/jira/browse/INFRA-12440 If there are no objections here on the dev list, I would like to move the "flume" and "redis" streaming connector to Bahir. Once they are in, and the file structure at

[jira] [Created] (FLINK-4407) Implement the trigger DSL

Kostas Kloudas created FLINK-4407: - Summary: Implement the trigger DSL Key: FLINK-4407 URL: https://issues.apache.org/jira/browse/FLINK-4407 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4406) Implement job master registration at resource manager

Wenlong Lyu created FLINK-4406: -- Summary: Implement job master registration at resource manager Key: FLINK-4406 URL: https://issues.apache.org/jira/browse/FLINK-4406 Project: Flink Issue Type: