[jira] [Created] (FLINK-2833) Unstage Gelly and Module refactoring

Vasia Kalavri created FLINK-2833: Summary: Unstage Gelly and Module refactoring Key: FLINK-2833 URL: https://issues.apache.org/jira/browse/FLINK-2833 Project: Flink Issue Type: Task

Re: [DISCUSS] flink-external

+1 for Vasia's suggestion. It's a good and pragmatic start. Putting such contributions into an Apache-managed repository might have some side effects. 2015-10-08 12:15 GMT+02:00 Vasiliki Kalavri : > How about, for now, we simply create a page where we gather

Re: [DISCUSS] flink-external

+1 for your pragmatic approach, Vasia. A simple collection of third party software using Flink should be enough for now; of course, outside the Apache realm. On Thu, Oct 8, 2015 at 12:45 PM, Chiwan Park wrote: > +1 for Vasia’s suggestion. From a long-term perspective, the

Re: [DISCUSS] flink-external

How about, for now, we simply create a page where we gather links/short descriptions of all these contributions and let the maintenance and dependency management to the tool/library creators? This way we will at least have these contributions in one place and link to them somewhere from the

Re: [DISCUSS] flink-external

Thanks for the feedback. I think, the repository does not need to build on a single Flink release. From my point of view, there should be a single parent module that contains *independent modules* for each extension/library (there should be no "cross dependencies" between the modules and each

Re: [DISCUSS] flink-external

Should we add a new page at Flink project web page? On 10/08/2015 12:56 PM, Maximilian Michels wrote: > +1 for your pragmatic approach, Vasia. A simple collection of third > party software using Flink should be enough for now; of course, > outside the Apache realm. > > On Thu, Oct 8, 2015 at

Re: [DISCUSS] flink-external

+1 for Vasia’s suggestion. From a long-term perspective, the site like Spark Packages [1] would be helpful to manage external contribution. [1] http://spark-packages.org > On Oct 8, 2015, at 12:28 PM, Matthias J. Sax wrote: > > Thanks for the feedback. > > I think, the

Re: Extending and improving our "How to contribute" page

Hi everybody, I merged our new contribution guidelines a few minutes ago. I'd like to emphasize that these rules do not have any effect, if nobody follows them. So please follow our contribution rules and make others aware of them as well. Specifically - pay attention that all PRs are backed by

Re: Extending and improving our "How to contribute" page

+1 Great! On 10/08/2015 01:14 PM, Fabian Hueske wrote: > Hi everybody, > > I merged our new contribution guidelines a few minutes ago. > > I'd like to emphasize that these rules do not have any effect, if nobody > follows them. > So please follow our contribution rules and make others aware of

Re: [DISCUSS] flink-external

Hi Matthias, Thanks for bringing up this idea. Actually, it has been discussed a couple of times on the mailing list whether we should have a central place for third-party extensions/contributions/libraries. This could either be something package-based or, like you proposed, another repository.

[jira] [Created] (FLINK-2834) Global round-robin for temporary directories

Greg Hogan created FLINK-2834: - Summary: Global round-robin for temporary directories Key: FLINK-2834 URL: https://issues.apache.org/jira/browse/FLINK-2834 Project: Flink Issue Type: Improvement

No error on dangling dataflow parts

Hi, I just hit a problem in Storm Compatibility: https://issues.apache.org/jira/browse/FLINK-2837 If a bolt has multiple inputs, the topology is not translated correctly into a Flink streaming program. The point is, that the Flink program can be executed without an error, even if the assembled

Re: No error on dangling dataflow parts

Well. This behavior would also be kind of strange... (at least to me) On 10/08/2015 04:22 PM, Aljoscha Krettek wrote: > Hi, > I think Flink does in fact not drop the dangling parts. In streaming it is > allowed to have dangling operators that are not sinks. They are executed and > the output is

Re: No error on dangling dataflow parts

What do you mean? The current behavior is strange or the other way round would be strange? I think it is in line with what other Stream Processing Systems provide. For example Storm and Google Dataflow behave similarly. > On 08 Oct 2015, at 16:25, Matthias J. Sax wrote: > >

[jira] [Created] (FLINK-2835) Update Documenation for new Streaming API

Matthias J. Sax created FLINK-2835: -- Summary: Update Documenation for new Streaming API Key: FLINK-2835 URL: https://issues.apache.org/jira/browse/FLINK-2835 Project: Flink Issue Type: Task

Re: [DISCUSS] flink-external

IMHO we can do that. There should be a disclaimer that the third party software is not officially supported. On Thu, Oct 8, 2015 at 2:25 PM, Matthias J. Sax wrote: > Should we add a new page at Flink project web page? > > On 10/08/2015 12:56 PM, Maximilian Michels wrote: >> +1

[jira] [Created] (FLINK-2836) Cyclic Topologies not supported

Matthias J. Sax created FLINK-2836: -- Summary: Cyclic Topologies not supported Key: FLINK-2836 URL: https://issues.apache.org/jira/browse/FLINK-2836 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-2837) FlinkTopologyBuilder cannot handle multiple input streams

Matthias J. Sax created FLINK-2837: -- Summary: FlinkTopologyBuilder cannot handle multiple input streams Key: FLINK-2837 URL: https://issues.apache.org/jira/browse/FLINK-2837 Project: Flink

Re: Iteration feedback partitioning does not work properly

Ok, I see your point. But I think there will be problems no matter what parallelism is chosen for the iteration source/sink. If the parallelism of the head is chosen then there will be an implicit rebalance from the operation right before the iteration to the iteration head. I think this should

Re: No error on dangling dataflow parts

I just skipped over the discussion. If I got it right, the question was about if a dedicated sink operator must be the last in a flow or not. In the case I described below, it is about distinct flows in a single program. This was not discussed in the PR (or did I miss it?). For sink/no-sink I

Re: No error on dangling dataflow parts

Dropping would be less strange. However, raising an exception would be natural (at least to me) On 10/08/2015 04:30 PM, Aljoscha Krettek wrote: > What do you mean? The current behavior is strange or the other way round > would be strange? > > I think it is in line with what other Stream

Re: Iteration feedback partitioning does not work properly

For me as an outsider to the iterations, I would say that both approaches are in some way tricky with some unexpected behavior. Parallelism implicitly from the predecessor (input) or the successor (head task - what happens if there are multiple with different parallelism?) can confuse in either

Re: Iteration feedback partitioning does not work properly

I agree that there are many things that needs to be figured out properly for iterations, and I am okay with postponing them for the next release if we want to get this one out quickly. The only problem is that this probably breaks the SAMOA connector. Paris can you confirm this? Stephan Ewen

Re: Extending and improving our "How to contribute" page

Thanks again for leading this effort, Fabian - Henry On Thursday, October 8, 2015, Fabian Hueske wrote: > Hi everybody, > > I merged our new contribution guidelines a few minutes ago. > > I'd like to emphasize that these rules do not have any effect, if nobody > follows

Re: Extending and improving our "How to contribute" page

Great, thanks Fabian! On Thu, Oct 8, 2015 at 5:28 PM, Henry Saputra wrote: > Thanks again for leading this effort, Fabian > > - Henry > > On Thursday, October 8, 2015, Fabian Hueske wrote: > > > Hi everybody, > > > > I merged our new contribution

Re: No error on dangling dataflow parts

We had some discussion on the Jira Issue when I changed the translation from operators to StreamGraph: https://issues.apache.org/jira/browse/FLINK-2398 There are arguments for both ways of doing it, I'm not partial towards any solution but if you want this changed you can start a discussion. (If

Re: Iteration feedback partitioning does not work properly

Yes it does break it since it is based on backwards partitioning preservation which was the case before Aljischa’s refactoring. I will focus on a 0.10 patch for the samoa connector right after the 0.10 release to see how we can do this. To be honest the whole thing confuses me a bit. From my

Re: Iteration feedback partitioning does not work properly

The feedback tuples might get rebalanced but the normal input should not. But still the main problem is the fact that partitioning is not handled transparently, and actually does not work when you set the way you expect. Gyula Aljoscha Krettek ezt írta (időpont: 2015. okt.