[jira] [Created] (FLINK-4690) Replace SlotAllocationFuture with flink's own future

Kurt Young created FLINK-4690: - Summary: Replace SlotAllocationFuture with flink's own future Key: FLINK-4690 URL: https://issues.apache.org/jira/browse/FLINK-4690 Project: Flink Issue Type:

Re: Flink: How to handle external app configuration changes in flink

Hi Jamie, Thanks a lot for the response. Appreciate your help. Regards, Govind On Mon, Sep 26, 2016 at 3:26 AM, Jamie Grier wrote: > Hi Govindarajan, > > Typically the way people do this is to create a stream of configuration > changes and consume this like any other

[jira] [Created] (FLINK-4689) Implement a simple slot provider for the new job manager

Kurt Young created FLINK-4689: - Summary: Implement a simple slot provider for the new job manager Key: FLINK-4689 URL: https://issues.apache.org/jira/browse/FLINK-4689 Project: Flink Issue Type:

[DISCUSS] Mesos Dispatcher (FF2016)

Hello, The code I presented at FF2016 represents a 'status-quo' approach to realizing a specific scenario - "mesos-session.sh". But the final solution will involve CLI changes and the full realization of a dispatcher, which conflicts with FLIP-6. We should advance the client/dispatcher

[jira] [Created] (FLINK-4688) Optimizer hangs for hours when optimizing complex plans

Fabian Hueske created FLINK-4688: Summary: Optimizer hangs for hours when optimizing complex plans Key: FLINK-4688 URL: https://issues.apache.org/jira/browse/FLINK-4688 Project: Flink Issue

[jira] [Created] (FLINK-4687) Add getAddress method to RpcService

Till Rohrmann created FLINK-4687: Summary: Add getAddress method to RpcService Key: FLINK-4687 URL: https://issues.apache.org/jira/browse/FLINK-4687 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-4686) Add possibility to get column names

Timo Walther created FLINK-4686: --- Summary: Add possibility to get column names Key: FLINK-4686 URL: https://issues.apache.org/jira/browse/FLINK-4686 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-4685) Gather operator checkpoint durations data sizes from the runtime

Stephan Ewen created FLINK-4685: --- Summary: Gather operator checkpoint durations data sizes from the runtime Key: FLINK-4685 URL: https://issues.apache.org/jira/browse/FLINK-4685 Project: Flink

[jira] [Created] (FLINK-4684) Remove obsolete classloader from CheckpointCoordinator

Stephan Ewen created FLINK-4684: --- Summary: Remove obsolete classloader from CheckpointCoordinator Key: FLINK-4684 URL: https://issues.apache.org/jira/browse/FLINK-4684 Project: Flink Issue

Re: Flink JDBCOutputFormat - Flush last batch enhancement

* setting the batch interval _to 1_ On 26.09.2016 15:25, Chesnay Schepler wrote: Hello Swapnil, setting the batch interval should be pretty much equivalent to having a streaming jdbc connector. Regards, Chesnay On 26.09.2016 13:21, Swapnil Chougule wrote: Hi Stephen/Chesnay, I have used

Ynt: No support for request PutMappingRequest

Hi Aljoscha, We are trying to add sortable feature for elasticsearch. To do this, we need to add mapping to index. We try to sort some fields in elasticsearch. To make it our json format should like this; "tweet": { "type": "string", "analyzer": "english", "fields": { "raw": { "type":

[jira] [Created] (FLINK-4678) Add SessionRow row-windows for streaming tables (FLIP-11)

Fabian Hueske created FLINK-4678: Summary: Add SessionRow row-windows for streaming tables (FLIP-11) Key: FLINK-4678 URL: https://issues.apache.org/jira/browse/FLINK-4678 Project: Flink

[jira] [Created] (FLINK-4683) Add SlideRow row-windows for batch tables

Fabian Hueske created FLINK-4683: Summary: Add SlideRow row-windows for batch tables Key: FLINK-4683 URL: https://issues.apache.org/jira/browse/FLINK-4683 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-4682) Add TumbleRow row-windows for batch tables.

Fabian Hueske created FLINK-4682: Summary: Add TumbleRow row-windows for batch tables. Key: FLINK-4682 URL: https://issues.apache.org/jira/browse/FLINK-4682 Project: Flink Issue Type:

[jira] [Created] (FLINK-4681) Add SessionRow row-windows for batch tables.

Fabian Hueske created FLINK-4681: Summary: Add SessionRow row-windows for batch tables. Key: FLINK-4681 URL: https://issues.apache.org/jira/browse/FLINK-4681 Project: Flink Issue Type:

[jira] [Created] (FLINK-4680) Add SlidingRow row-windows for streaming tables

Fabian Hueske created FLINK-4680: Summary: Add SlidingRow row-windows for streaming tables Key: FLINK-4680 URL: https://issues.apache.org/jira/browse/FLINK-4680 Project: Flink Issue Type:

[jira] [Created] (FLINK-4679) Add TumbleRow row-windows for streaming tables

Fabian Hueske created FLINK-4679: Summary: Add TumbleRow row-windows for streaming tables Key: FLINK-4679 URL: https://issues.apache.org/jira/browse/FLINK-4679 Project: Flink Issue Type:

Re: No support for request PutMappingRequest

Hi, I think PutMappingRequest is a request that can only be sent using IndicesAdminClient. In my understanding this is an administrative command that isn't related to actually storing data in an index. What are you trying to store with the PutMappingRequest? Cheers, Aljoscha On Mon, 26 Sep 2016

Re: Flink JDBCOutputFormat - Flush last batch enhancement

Hello Swapnil, setting the batch interval should be pretty much equivalent to having a streaming jdbc connector. Regards, Chesnay On 26.09.2016 13:21, Swapnil Chougule wrote: Hi Stephen/Chesnay, I have used JDBCOutputFormat from batch connectors for my streaming use case as I didn't find

Ynt: No support for request PutMappingRequest

Hi, I am sending the error message below; java.lang.IllegalArgumentException: No support for request [org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest@3ed05d8b] at org.elasticsearch.action.bulk.BulkRequest.add(BulkRequest.java:107) ~[elasticsearch-2.3.5.jar:2.3.5] at

Re: [DISCUSS] FLIP-11: Table API Stream Aggregations

Hi everybody, Timo proposed our FLIP-11 a bit more than three weeks ago. I will update the status of the FLIP to accepted. Thanks, Fabian 2016-09-19 9:16 GMT+02:00 Timo Walther : > Hi Jark, > > yes I think enough time has passed. We can start implementing the changes. >

[DISCUSS] add netty tcp/restful pushed source support

Hi, all 1.In order to support end-to-end pushed source, I create FLINK-4630. I want to know whether is this idea worth? --- When source stream get start, listen a provided tcp port, receive stream data from user data

[jira] [Created] (FLINK-4677) Jars with no job executions produces NullPointerException in ClusterClient

Maximilian Michels created FLINK-4677: - Summary: Jars with no job executions produces NullPointerException in ClusterClient Key: FLINK-4677 URL: https://issues.apache.org/jira/browse/FLINK-4677

Re: [DISCUSS] Merge batch and stream connector modules

Thanks everybody for your comments. I opened FLINK-4676 [1] for merging the connector modules. [1] https://issues.apache.org/jira/browse/FLINK-4676 2016-09-26 13:17 GMT+02:00 Robert Metzger : > +1 good suggestion. > > On Mon, Sep 26, 2016 at 1:03 PM, Stephan Ewen

[jira] [Created] (FLINK-4676) Merge flink-batch-con

Fabian Hueske created FLINK-4676: Summary: Merge flink-batch-con Key: FLINK-4676 URL: https://issues.apache.org/jira/browse/FLINK-4676 Project: Flink Issue Type: Task Reporter:

Re: Flink JDBCOutputFormat - Flush last batch enhancement

Hi Stephen/Chesnay, I have used JDBCOutputFormat from batch connectors for my streaming use case as I didn't find jdbc connector from streaming connectors. If it is not there, may we have jdbc connector for streaming use cases? Thanks, Swapnil On Mon, Sep 26, 2016 at 3:32 PM, Chesnay Schepler

答复: 答复: [discuss] merge module flink-yarn and flink-yarn-test

Hi，Maximilian Michels Thank for your reply. In order to test submit Flink job to yarn cluster, that is testing client and session cli, split unit test from flink-yarn. First of all, such yarn cluster unit test should be rename 'flink-yarn-tests' as 'flink-yarn-cluster-test', and move to

Re: [DISCUSS] Merge batch and stream connector modules

+1 good suggestion. On Mon, Sep 26, 2016 at 1:03 PM, Stephan Ewen wrote: > The module would have both dependencies, but both are provided anyways, so > that would not be much of an issue, I think. > > On Mon, Sep 26, 2016 at 12:25 PM, Till Rohrmann >

Re: [DISCUSS] Merge batch and stream connector modules

The module would have both dependencies, but both are provided anyways, so that would not be much of an issue, I think. On Mon, Sep 26, 2016 at 12:25 PM, Till Rohrmann wrote: > I think this only holds true for modules which depend on the batch or > streaming counter part,

Re: Exception from in-progress implementation of Python API bulk iterations

Hello Geoffrey, i could not reproduce this issue with the commits and plan you provided. I tried out both the FLINK-4098 and bulk-iterations branches (and reverted back to the specified commits) and built Flink from scratch. Could you double check that the code you provided produces the

Re: No support for request PutMappingRequest

Hi Ozan, I'm not super experienced with Flink's elasticsearch connector, but could you post the complete stack trace to figure out where the problem comes from? I've also pulled in Aljoscha, the original author of the elasticsearch sink. Maybe he can give you a detailed answer. Cheers, Till On

Re: Flink: How to handle external app configuration changes in flink

Hi Govindarajan, Typically the way people do this is to create a stream of configuration changes and consume this like any other stream. For the specific case of filtering for example you may have a data stream and a stream of filters that you want to run the data through. The typically

Re: [DISCUSS] Merge batch and stream connector modules

I think this only holds true for modules which depend on the batch or streaming counter part, respectively. We could refactor these modules by pulling out common types which are independent of streaming/batch and are used by the batch and streaming module. Cheers, Till On Fri, Sep 23, 2016 at

Re: FailureRate Restart Strategy is not picked from Config file

Hi Deepak, are you running Flink streaming jobs with checkpointing enabled? In this case Flink will check if you've set a restart strategy at your job and if not it will set the fixed delay restart strategy. This will effectively overwrite the default restart strategy which you define in the

Re: Questions on flink

Hi Govindarajan, I've put some answers in-line below.. On Sat, Sep 24, 2016 at 7:32 PM, Govindarajan Srinivasaraghavan < govindragh...@gmail.com> wrote: > Hi, > > I'm working on apache flink for data streaming and I have few questions. > Any help is greatly appreciated. Thanks. > > 1) Are there

Re: Flink JDBCOutputFormat - Flush last batch enhancement

The JDBCOutputFormat writes records in batches, that's what he is referring to. On 26.09.2016 11:48, Stephan Ewen wrote: Hi! I am not sure I understand what you want to do, but here are some comments: - There is no "batching" in Flink's streaming API, not sure what you are referring to in

Re: Flink JDBCOutputFormat - Flush last batch enhancement

Hi! I am not sure I understand what you want to do, but here are some comments: - There is no "batching" in Flink's streaming API, not sure what you are referring to in with the "last batch" - JDBC connections are not closed between windows, they remain open as long as the operator is open.

Re: 答复: [discuss] merge module flink-yarn and flink-yarn-test

Hello Jinkui Shi, Due to the nature of most of the Yarn tests, we need them to be in a separate module. More concretely, these tests have a dependency on 'flink-dist' because they need to deploy the Flink fat jar to the Yarn tests cluster. The fat jar also contains the 'flink-yarn' code. Thus,

Flink JDBCOutputFormat - Flush last batch enhancement

Hi Team, Can we handle one case in connector JDBCOutputFormat to update last batch (might be batch count is less than batch interval) without closing jdbc connection? During use case of my streaming project, I am updating jdbc sink (mysql db) after every window. Case : Say I have 450 queries to