Re: TopologyBuilder throws java.lang.ExceptionInInitializerError

Hi, I am not sure if this is useful, I once had similar issue. Are you including storm-core dependency in your project? flink-storm dependency doesn't need any specific storm-core dependency. Just including flink-storm and removing storm-core dependency worked for me. This has more information,

Re: XGBoost on DataFlow and Flink

Thanks! I am not aware of SrcOperator before. Then yes things can be done. About multi-threading issue, I am looking for more principled API to specify the resources requirement, e.g. the slots in this stage needs 4 GPU cores and 1 GPU. So the resource allocator can be aware of that. We have

TopologyBuilder throws java.lang.ExceptionInInitializerError

Hi, I am converting a storm topology to Flink-storm topology using the flink-storm dependency. When I run my code the FlinkTopologyBuilder eventually calls createTopology method in TopologyBuilder and throws the error at the following highlighted line:- public StormTopology createTopology()

Re: [streaming, scala] Scala DataStream#addSink returns Java DataStreamSink

Hey, I think we came to the agreement that this PR is not mergeable right now, so I am closing it. I personally find it inconsistent to not have the fully API mirrored in Scala though, but this is something that we can revisit when prepairing 2.0. Best, Marton On Mon, Mar 14, 2016 at 8:14 PM,

Re: [streaming, scala] Scala DataStream#addSink returns Java DataStreamSink

+1 for considering this as a bug. However, I do realize that API compatibility is extremely important when you commit to it, even if present behavior is not ideal. Silly idea (which is only applicable to Scala APIs, unfortunately): I'm currently working on a set of API extensions to solve

Re: XGBoost on DataFlow and Flink

Hi Tianqi, dmlc looks really cool and it would be great to integrate it with Flink. As far as I understood your requirements, I think that you can already implement most of it on Flink. For example, starting a special container which does not receive any input could be a specialized

[jira] [Created] (FLINK-3613) Add standard deviation to list of Aggregations

Todd Lisonbee created FLINK-3613: Summary: Add standard deviation to list of Aggregations Key: FLINK-3613 URL: https://issues.apache.org/jira/browse/FLINK-3613 Project: Flink Issue Type:

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

Hi Maximilian, Thanks for your response. I will wait for the update. On Monday, March 14, 2016, Maximilian Michels wrote: > Hi Deepak, > > We'll look more into this problem this week. Until now we considered it a > configuration issue if the bind address was not externally

[jira] [Created] (FLINK-3612) Fix/adjust Table API examples

Vasia Kalavri created FLINK-3612: Summary: Fix/adjust Table API examples Key: FLINK-3612 URL: https://issues.apache.org/jira/browse/FLINK-3612 Project: Flink Issue Type: Bug

Re: zipWithIndex in Python API

I'm a Python guru; if it doesn't have a Python API, I'll likely help make one :) Work is bad this week but I'm planning to get started on this next week! Shannon On 3/14/16 5:37 AM, Robert Metzger wrote: Hi Shannon, I'm happy to see some community engagement on our Python APIs! On Fri, Mar

Re: [DISCUSSION] Database state backend

On Mon, Mar 14, 2016 at 11:20 AM, Gyula Fóra wrote: > We developed (and contributed) the DB state backend to be able to handle > large state sizes, but have moved to RocksDB for much better read/write > performance. I have a pending PR with some improvements to the MySQL >

Re: [streaming, scala] Scala DataStream#addSink returns Java DataStreamSink

I agree with Aljoscha on this one, because `DataStreamSink` only contains setters which are compatible with the Scala API. On Mon, Mar 14, 2016 at 11:02 AM, Aljoscha Krettek wrote: > By the way, I don’t think it’s a bug that addSink() returns the Java > DataStreamSink.

[DISCUSSION] Database state backend

Hello everyone! I would like to start a discussion regarding the future of the Database state backend for the streaming API. The main question is whether we want to keep this as a flink-contrib moduel and continue development as part of Flink or should we move it to an outside library. Just as a

Re: GSoC Project Proposal Draft: Code Generation in Serializers

Hi, I have updated this draft to include preliminary benchmarks, mentioned the interaction of annotations with savepoints, extended it with a timeline, and some notes about scala case classes. Regards, Gábor On 9 March 2016 at 16:12, Gábor Horváth wrote: > Hi! > > As far

Re: [streaming, scala] Scala DataStream#addSink returns Java DataStreamSink

By the way, I don’t think it’s a bug that addSink() returns the Java DataStreamSink. Having a Scala specific version of a DataStreamSink would not add functionality in this place, just code bloat. > On 14 Mar 2016, at 10:05, Fabian Hueske wrote: > > Yes, we will have more of

Re: zipWithIndex in Python API

Hi Shannon, I'm happy to see some community engagement on our Python APIs! On Fri, Mar 11, 2016 at 6:32 PM, Chesnay Schepler wrote: > The subtaskIndex is not currently exposed to the python operator. > > Fortunately this can be changed very easily: > On the java side,

Re: Flink-1.0.0 JobManager is not running in Docker Container on AWS

Hi Deepak, We'll look more into this problem this week. Until now we considered it a configuration issue if the bind address was not externally reachable. However, one might not always have the possibility to change this network configuration. Looking further, it is actually possible to let the