Re: Consuming data from dynamoDB streams to flink

Hi! I think it would be definitely nice to have this feature. No actual previous work has been made on this issue, but AFAIK, we should be able to build this on top of the FlinkKinesisConsumer. Whether this should live within the Kinesis connector module or an independent module of its own is

Re: FLINK-6222

Hi, Craig The patch you attached there seems do not follow flink community specification. Could you please link to a Github pull request in there ? Cheers Minglei > 在 2018年6月28日，上午3:56，Foster, Craig 写道： > > Pinging. Is it possible for someone to take a look at this or is this message >

[jira] [Created] (FLINK-9685) Flink should support hostname-substitution for security.kerberos.login.principal

Ethan Li created FLINK-9685: --- Summary: Flink should support hostname-substitution for security.kerberos.login.principal Key: FLINK-9685 URL: https://issues.apache.org/jira/browse/FLINK-9685 Project: Flink

[jira] [Created] (FLINK-9684) HistoryServerArchiveFetcher not working properly with secure hdfs cluster

Ethan Li created FLINK-9684: --- Summary: HistoryServerArchiveFetcher not working properly with secure hdfs cluster Key: FLINK-9684 URL: https://issues.apache.org/jira/browse/FLINK-9684 Project: Flink

Re: FLINK-6222

Pinging. Is it possible for someone to take a look at this or is this message going into a black hole? Thanks. From: "Foster, Craig" Date: Thursday, June 14, 2018 at 11:22 AM To: "dev@flink.apache.org" Subject: FLINK-6222 Greetings: I’m wondering how to get someone to review my patch? The

[jira] [Created] (FLINK-9683) inconsistent behaviors when setting historyserver.archive.fs.dir

Ethan Li created FLINK-9683: --- Summary: inconsistent behaviors when setting historyserver.archive.fs.dir Key: FLINK-9683 URL: https://issues.apache.org/jira/browse/FLINK-9683 Project: Flink Issue

Re: [Flink-9407] Question about proposed ORC Sink !

Thanks @zhangminglei and @Fabian for confirming. Even I looked at the ORC parsing code and it seems that using type is mandatory for now. Thanks, Sagar On Wed, Jun 27, 2018 at 12:59 AM, Fabian Hueske wrote: > Hi Sagar, > > That's more a question for the ORC community, but AFAIK, the

Re: Consuming data from dynamoDB streams to flink

Thanks Fabian for the suggestion. *Ying Xu* Software Engineer 510.368.1252 <+15103681252> [image: Lyft] On Wed, Jun 27, 2018 at 2:01 AM, Fabian Hueske wrote: > Hi Ying, > > I'm not aware of any effort for this issue. > You could check with the assigned contributor in

[jira] [Created] (FLINK-9682) Add setDescription to execution environment and display it in the UI

Elias Levy created FLINK-9682: - Summary: Add setDescription to execution environment and display it in the UI Key: FLINK-9682 URL: https://issues.apache.org/jira/browse/FLINK-9682 Project: Flink

[jira] [Created] (FLINK-9681) Make sure minRetentionTime not equal to maxRetentionTime

Hequn Cheng created FLINK-9681: -- Summary: Make sure minRetentionTime not equal to maxRetentionTime Key: FLINK-9681 URL: https://issues.apache.org/jira/browse/FLINK-9681 Project: Flink Issue

[jira] [Created] (FLINK-9680) Reduce heartbeat timeout for E2E tests

Chesnay Schepler created FLINK-9680: --- Summary: Reduce heartbeat timeout for E2E tests Key: FLINK-9680 URL: https://issues.apache.org/jira/browse/FLINK-9680 Project: Flink Issue Type:

[jira] [Created] (FLINK-9679) Implement AvroSerializationSchema

Yazdan Shirvany created FLINK-9679: -- Summary: Implement AvroSerializationSchema Key: FLINK-9679 URL: https://issues.apache.org/jira/browse/FLINK-9679 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-9678) Remove hard-coded sleeps in HA E2E test

Chesnay Schepler created FLINK-9678: --- Summary: Remove hard-coded sleeps in HA E2E test Key: FLINK-9678 URL: https://issues.apache.org/jira/browse/FLINK-9678 Project: Flink Issue Type:

[jira] [Created] (FLINK-9677) RestClient fails for large uploads

Chesnay Schepler created FLINK-9677: --- Summary: RestClient fails for large uploads Key: FLINK-9677 URL: https://issues.apache.org/jira/browse/FLINK-9677 Project: Flink Issue Type: Bug

Re: Streaming

Forward shiming mail to Aitozi. Aitozi We are using hyperloglog to count daily uv, but it only provided an approximate value. I also tried the count distinct in flink table without window, but need to set the retention time. However, the time resolution of this operator is 1 millisecond, so

Re: Streaming

To aitozi. Cheers Minglei > 在 2018年6月27日，下午5:46，shimin yang 写道： > > Aitozi > > We are using hyperloglog to count daily uv, but it only provided an > approximate value. I also tried the count distinct in flink table without > window, but need to set the retention time. > > However, the

[jira] [Created] (FLINK-9676) Deadlock during canceling task and recycling exclusive buffer

zhijiang created FLINK-9676: --- Summary: Deadlock during canceling task and recycling exclusive buffer Key: FLINK-9676 URL: https://issues.apache.org/jira/browse/FLINK-9676 Project: Flink Issue

[jira] [Created] (FLINK-9675) Avoid FileInputStream/FileOutputStream

Ted Yu created FLINK-9675: - Summary: Avoid FileInputStream/FileOutputStream Key: FLINK-9675 URL: https://issues.apache.org/jira/browse/FLINK-9675 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-9674) Remove 65s sleep in QueryableState E2E test

Chesnay Schepler created FLINK-9674: --- Summary: Remove 65s sleep in QueryableState E2E test Key: FLINK-9674 URL: https://issues.apache.org/jira/browse/FLINK-9674 Project: Flink Issue Type:

[jira] [Created] (FLINK-9673) Improve State efficiency of bounded OVER window operators

Fabian Hueske created FLINK-9673: Summary: Improve State efficiency of bounded OVER window operators Key: FLINK-9673 URL: https://issues.apache.org/jira/browse/FLINK-9673 Project: Flink

Re: Consuming data from dynamoDB streams to flink

Hi Ying, I'm not aware of any effort for this issue. You could check with the assigned contributor in Jira if there is some previous work. Best, Fabian 2018-06-26 9:46 GMT+02:00 Ying Xu : > Hello Flink dev: > > We have a number of use cases which involves pulling data from DynamoDB > streams

[jira] [Created] (FLINK-9672) Fail fatally if we cannot submit job on added JobGraph signal

Till Rohrmann created FLINK-9672: Summary: Fail fatally if we cannot submit job on added JobGraph signal Key: FLINK-9672 URL: https://issues.apache.org/jira/browse/FLINK-9672 Project: Flink

[jira] [Created] (FLINK-9671) Add configuration to enable task manager isolation.

Renjie Liu created FLINK-9671: - Summary: Add configuration to enable task manager isolation. Key: FLINK-9671 URL: https://issues.apache.org/jira/browse/FLINK-9671 Project: Flink Issue Type: New

[jira] [Created] (FLINK-9670) Introduce slot manager factory

Renjie Liu created FLINK-9670: - Summary: Introduce slot manager factory Key: FLINK-9670 URL: https://issues.apache.org/jira/browse/FLINK-9670 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-9669) Introduce task manager assignment store

Renjie Liu created FLINK-9669: - Summary: Introduce task manager assignment store Key: FLINK-9669 URL: https://issues.apache.org/jira/browse/FLINK-9669 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-9668) In some case Trigger.onProcessingTime don't exectue

qi quan created FLINK-9668: -- Summary: In some case Trigger.onProcessingTime don't exectue Key: FLINK-9668 URL: https://issues.apache.org/jira/browse/FLINK-9668 Project: Flink Issue Type: Bug

Re: [Flink-9407] Question about proposed ORC Sink !

Hi Sagar, That's more a question for the ORC community, but AFAIK, the top-level type is always a struct because it needs to wrap the fields, e.g., struct(name:string, age:int) Best, Fabian 2018-06-26 22:38 GMT+02:00 sagar loke : > @zhangminglei, > > Question about the schema for ORC format: >

[jira] [Created] (FLINK-9667) Scala docs are only generated for flink-scala

Chesnay Schepler created FLINK-9667: --- Summary: Scala docs are only generated for flink-scala Key: FLINK-9667 URL: https://issues.apache.org/jira/browse/FLINK-9667 Project: Flink Issue

[jira] [Created] (FLINK-9666) short-circuit logic should be used in boolean contexts

lamber-ken created FLINK-9666: - Summary: short-circuit logic should be used in boolean contexts Key: FLINK-9666 URL: https://issues.apache.org/jira/browse/FLINK-9666 Project: Flink Issue Type: