[jira] [Created] (FLINK-11305) Allow developer to test flink without building the whole flink project

Jeff Zhang created FLINK-11305: -- Summary: Allow developer to test flink without building the whole flink project Key: FLINK-11305 URL: https://issues.apache.org/jira/browse/FLINK-11305 Project: Flink

Apply for dev authority

Hey, I’d like to apply for dev authority, Here is my jira account is : JaryZhen Thinks

[jira] [Created] (FLINK-11304) Typo in time attributes doc

Jaryzhen created FLINK-11304: Summary: Typo in time attributes doc Key: FLINK-11304 URL: https://issues.apache.org/jira/browse/FLINK-11304 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-11303) Utilizing Kafka headers for serialization and deserialization

Allen Wang created FLINK-11303: -- Summary: Utilizing Kafka headers for serialization and deserialization Key: FLINK-11303 URL: https://issues.apache.org/jira/browse/FLINK-11303 Project: Flink

Recovery problem in Flink 1.6.3

This is the first of two recovery problems I'm seeing running Flink 1.6.3 in Kubernetes. I'm posting them in separate messages for brevity and because the second is not directly related to the first. Any advice is appreciated. Setup: Flink 1.6.3 in Kubernetes

Re: ElasticSearch Connector

Hi, I'm afraid you cannot write to different indices using the table API ElasticSearch connector. Now I know why you wanted to go through datastream API. What you could do to transform from Row to JSON is to use org.apache.flink.formats.json.JsonRowSerializationSchema from flink-json. You just

[jira] [Created] (FLINK-11302) FlinkS3FileSystem uses an incorrect path for temporary files.

Artsem Semianenka created FLINK-11302: - Summary: FlinkS3FileSystem uses an incorrect path for temporary files. Key: FLINK-11302 URL: https://issues.apache.org/jira/browse/FLINK-11302 Project:

Re: [DISCUSS] Support Interactive Programming in Flink Table API

I spoke to Piotr a little bit offline and I wanted to comment with a summary of our discussion and what I believe is most intuitive cache model from a users perspective. (I am making up some class names here, not looking to bike shed feel free to change the names how ever you see fit). A

[jira] [Created] (FLINK-11301) Travis failed: No output has been received in the last 10m0s

Hequn Cheng created FLINK-11301: --- Summary: Travis failed: No output has been received in the last 10m0s Key: FLINK-11301 URL: https://issues.apache.org/jira/browse/FLINK-11301 Project: Flink

Re: [DISCUSS] Bot for stale PRs on GitHub

> The bot will remind both reviewers and contributors that they have to be active on a PR, I found that useful on some PRs that I had open at Beam I don't think we really want every contributor bumping their PR regularly. This will create unbearable noise and, if they actually update it, will

Re: [DISCUSS] Bot for stale PRs on GitHub

For reference, this is the older staleness discussion: https://lists.apache.org/thread.html/d53bee8431776f38ebaf8f5678b1ffd9513cd65ce15d821bbdca95aa@%3Cdev.flink.apache.org%3E

Re: [DISCUSS] Bot for stale PRs on GitHub

Without any new argument for doing so, I'm still against it. On 10.01.2019 09:54, Aljoscha Krettek wrote: Hi, I know we had similar discussions in the past but I’d like to bring up this topic again. What do you think about adding a stale bot (https://probot.github.io/apps/stale/

Re: [DISCUSS] Dropping flink-storm?

+1 from my side as well. I would assume that most Bolts that are supported by our current wrappers can be easily converted into respective Flink functions. Fabian Am Do., 10. Jan. 2019 um 10:35 Uhr schrieb Kostas Kloudas < k.klou...@da-platform.com>: > +1 to drop as well. > > On Thu, Jan 10,

Re: SQL codegen bytecode beyond 64KB.

Hi, Do you mind post your sql...? There are special cases that could result in this error.. -- Original -- From: "563280193"<563280...@qq.com>; Date: Thu, Jan 10, 2019 03:34 PM To: "dev";"user"; Subject: SQL codegen bytecode beyond 64KB. Hi all,

[jira] [Created] (FLINK-11300) Make MetricFetcher update interval configurable

Till Rohrmann created FLINK-11300: - Summary: Make MetricFetcher update interval configurable Key: FLINK-11300 URL: https://issues.apache.org/jira/browse/FLINK-11300 Project: Flink Issue

Re: ElasticSearch Connector

Hi, Sorry I am a beginner here. I am not really sure how to pack the dynamic indices here. the .index(test-ddmmyy) kind of indices here. I have set the watermark for my kafka table source, but not sure how this works on the the ElasticSearch Sink. Pasted my sample code below:

Re: [DISCUSS] Dropping flink-storm?

+1 to drop as well. On Thu, Jan 10, 2019 at 10:15 AM Ufuk Celebi wrote: > +1 to drop. > > I totally agree with your reasoning. I like that we tried to keep it, > but I don't think the maintenance overhead would be justified. > > – Ufuk > > On Wed, Jan 9, 2019 at 4:09 PM Till Rohrmann wrote: >

Re: ElasticSearch Connector

You can use flink to manipulate the data by using TimeCharacteristic.EventTime[1] and set Watermark. Then if you have a lag or other issue the data will be insert to the correct Indexes in elastic. More specific way to implement it with kafka[2] 1.

Re: [DISCUSS] Dropping flink-storm?

+1 to drop. I totally agree with your reasoning. I like that we tried to keep it, but I don't think the maintenance overhead would be justified. – Ufuk On Wed, Jan 9, 2019 at 4:09 PM Till Rohrmann wrote: > > With https://issues.apache.org/jira/browse/FLINK-10571, we will remove the > Storm

Re: ElasticSearch Connector

Hi David, thanks for the quick reply. I did try that. I am not sure how to push into rolling indices here. For example, i would maintain daily indices on ES. Based on the event time, i would like to classify the packets to appropriate indices. If there was some lag in the source kafka, and i get

[DISCUSS] Bot for stale PRs on GitHub

Hi, I know we had similar discussions in the past but I’d like to bring up this topic again. What do you think about adding a stale bot (https://probot.github.io/apps/stale/ ) to our Github Repo? This would automatically nag about stale PRs and close them

Re: ElasticSearch Connector

Hi Ramya, Have you tried writing to ES directly from table API? You can check the ES connector for table API here: https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/table/connect.html#elasticsearch-connector Best, Dawid On 10/01/2019 09:21, Ramya Ramamurthy wrote: > Hi, > > I am

ElasticSearch Connector

Hi, I am learning to Flink. With Flink 1.7.1, trying to read from Kafka and insert to ElasticSearch. I have a kafka connector convert the data to a Flink table. In order to insert into Elasticsearch, I have converted this table to a datastream, in order to be able to use the ElasticSearchSink.