Re: Flink Cassandra Sink - Setting TTL for the record

2018-06-04 Thread Rohan Thimmappa
Thanks. I was able get this working with insert statement with "using TTL" https://docs.datastax.com/en/cql/3.3/cql/cql_using/useExpireExample.html Rohan On Mon, Jun 4, 2018 at 11:45 AM, miki haiat wrote: > if you dont have this option in the conector API you need to implement a >

Fwd: [DISCUSS] FLIP-6 Problems

2018-06-04 Thread Renjie Liu
-- Forwarded message - From: Renjie Liu Date: Tue, Jun 5, 2018 at 10:43 AM Subject: [DISCUSS] FLIP-6 Problems To: user Hi: We've deployed flink 1.5.0 and tested the new cluster manager, it's really great for flink to be elastic. However we've also found some problems that

[jira] [Created] (FLINK-9524) NPE from ProcTimeBoundedRangeOver.scala

2018-06-04 Thread yan zhou (JIRA)
yan zhou created FLINK-9524: --- Summary: NPE from ProcTimeBoundedRangeOver.scala Key: FLINK-9524 URL: https://issues.apache.org/jira/browse/FLINK-9524 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-9523) Add Kafka examples for Flink Table/SQL API

2018-06-04 Thread Shuyi Chen (JIRA)
Shuyi Chen created FLINK-9523: - Summary: Add Kafka examples for Flink Table/SQL API Key: FLINK-9523 URL: https://issues.apache.org/jira/browse/FLINK-9523 Project: Flink Issue Type: Task

Re: [DISCUSS] Flink 1.6 features

2018-06-04 Thread Che Lui Shum
Hi Stephan, Will FLINK-7129 (Support dynamically changing CEP patterns) be included in 1.6? There were discussions about possibly including it in 1.6: http://mail-archives.apache.org/mod_mbox/flink-user/201803.mbox/%3cCAMq=ou7gru2o9jtowxn1lc1f7nkcxayn6a3e58kxctb4b50...@mail.gmail.com%3e Thanks,

[jira] [Created] (FLINK-9522) Rework Flink website

2018-06-04 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-9522: Summary: Rework Flink website Key: FLINK-9522 URL: https://issues.apache.org/jira/browse/FLINK-9522 Project: Flink Issue Type: Improvement

Flink Cassandra Sink - Setting TTL for the record

2018-06-04 Thread Rohan Thimmappa
Hi there, i am using timeseries data in cassandra by reading data from kafka. I didn't see any option set the TTL. Any help would be of great help! -- Thanks Rohan

Re: Ask for SQL using kafka in Flink

2018-06-04 Thread Shuyi Chen
Given the popularity of Flink SQL and Kafka as streaming source, I think we can add some examples of using Kafka[XXX]TableSource in flink-examples/flink-examples-table module. What do you guys think? Cheers Shuyi On Mon, Jun 4, 2018 at 12:57 AM, Timo Walther wrote: > Hi, > > as you can see in

[jira] [Created] (FLINK-9521) Add build plugin to package table example jar

2018-06-04 Thread vinoyang (JIRA)
vinoyang created FLINK-9521: --- Summary: Add build plugin to package table example jar Key: FLINK-9521 URL: https://issues.apache.org/jira/browse/FLINK-9521 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-9520) JarFileCreator does not support Java MethodReferences

2018-06-04 Thread Timo Walther (JIRA)
Timo Walther created FLINK-9520: --- Summary: JarFileCreator does not support Java MethodReferences Key: FLINK-9520 URL: https://issues.apache.org/jira/browse/FLINK-9520 Project: Flink Issue

[jira] [Created] (FLINK-9519) table example jar did not copy into example dir

2018-06-04 Thread vinoyang (JIRA)
vinoyang created FLINK-9519: --- Summary: table example jar did not copy into example dir Key: FLINK-9519 URL: https://issues.apache.org/jira/browse/FLINK-9519 Project: Flink Issue Type: Bug

Re: [DISCUSS] Flink 1.6 features

2018-06-04 Thread Till Rohrmann
Before removing the legacy code, I would still wait a bit and see what the user feedback is. The legacy mode is a good safety net against severe deployment regressions. Thus, it should be a very conscious decision to remove the code. As far as I know, there is currently nobody actively working on

[jira] [Created] (FLINK-9518) SSL setup Docs config example has wrong keys password

2018-06-04 Thread Yazdan Shirvany (JIRA)
Yazdan Shirvany created FLINK-9518: -- Summary: SSL setup Docs config example has wrong keys password Key: FLINK-9518 URL: https://issues.apache.org/jira/browse/FLINK-9518 Project: Flink

[jira] [Created] (FLINK-9517) Fixing broken links on CLI Docs

2018-06-04 Thread Yazdan Shirvany (JIRA)
Yazdan Shirvany created FLINK-9517: -- Summary: Fixing broken links on CLI Docs Key: FLINK-9517 URL: https://issues.apache.org/jira/browse/FLINK-9517 Project: Flink Issue Type: Improvement

Re:[DISCUSS] Flink 1.6 features

2018-06-04 Thread sihua zhou
Hi Stephan, could you please also consider the "Elastic Filter " feature discussioned in thread http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PROPOSAL-Introduce-Elastic-Bloom-Filter-For-Flink-td22430.html ? Best, Sihua On 06/4/2018 17:21,Stephan Ewen wrote: Hi Flink

[DISCUSS] Flink 1.6 features

2018-06-04 Thread Stephan Ewen
Hi Flink Community! The release of Apache Flink 1.5 has happened (yay!) - so it is a good time to start talking about what to do for release 1.6. *== Suggested release timeline ==* I would propose to release around *end of July* (that is 8-9 weeks from now). The rational behind that: There was

Re: [Discuss] Proposing FLIP-25 - Support User State TTL Natively in Flink

2018-06-04 Thread Fabian Hueske
Hi, I like the separation of visibility and clean-up. So far the design only addressed the clean-up aspect and aimed to have state accessible as long as possible, i.e., until it was cleared. We did not consider the use case of compliance. Support for strict visibility is a good idea, IMO.

[ANNOUNCE] Weekly community update #23

2018-06-04 Thread Till Rohrmann
Dear community, this is the weekly community update thread #23. Please post any news and updates you want to share with the community to this thread. # Releasing flink-shaded 4.0 The community is currently testing the RC 2 for flink-shaded 4.0 [1] which bumps the shaded Netty version to 4.1.24

Re: [TABLE][SQL] Unify UniqueKeyExtractor and DataStreamRetractionRules

2018-06-04 Thread Timo Walther
Hi Piotr, thanks for bringing up this discussion. I was not involved in the design discussions at that time but I also find the logic about upserts and retractions in multiple stages quite confusing. So in general +1 for simplification, however, by using a RelShuttle instead of rules we might

[jira] [Created] (FLINK-9516) Create wrapper with TTL logic for map state

2018-06-04 Thread Andrey Zagrebin (JIRA)
Andrey Zagrebin created FLINK-9516: -- Summary: Create wrapper with TTL logic for map state Key: FLINK-9516 URL: https://issues.apache.org/jira/browse/FLINK-9516 Project: Flink Issue Type:

[jira] [Created] (FLINK-9515) Create wrapper with TTL logic for list state

2018-06-04 Thread Andrey Zagrebin (JIRA)
Andrey Zagrebin created FLINK-9515: -- Summary: Create wrapper with TTL logic for list state Key: FLINK-9515 URL: https://issues.apache.org/jira/browse/FLINK-9515 Project: Flink Issue Type:

[jira] [Created] (FLINK-9514) Create wrapper with TTL logic for value state

2018-06-04 Thread Andrey Zagrebin (JIRA)
Andrey Zagrebin created FLINK-9514: -- Summary: Create wrapper with TTL logic for value state Key: FLINK-9514 URL: https://issues.apache.org/jira/browse/FLINK-9514 Project: Flink Issue Type:

[jira] [Created] (FLINK-9513) Wrap state binder with TTL logic

2018-06-04 Thread Andrey Zagrebin (JIRA)
Andrey Zagrebin created FLINK-9513: -- Summary: Wrap state binder with TTL logic Key: FLINK-9513 URL: https://issues.apache.org/jira/browse/FLINK-9513 Project: Flink Issue Type: Sub-task

Re: [Discuss] Proposing FLIP-25 - Support User State TTL Natively in Flink

2018-06-04 Thread sihua zhou
Hi andrey, Thanks for this doc! TBH, personally I prefer the approach you outlined in the doc over the previous one that purly based on timers. I think this approach looks very similar to the approach I outlined in this thread before, so it still face the challenges that @Bowen outlined, but

[jira] [Created] (FLINK-9512) StateDescriptor TTL config should be part of versioned meta info snapshot

2018-06-04 Thread Andrey Zagrebin (JIRA)
Andrey Zagrebin created FLINK-9512: -- Summary: StateDescriptor TTL config should be part of versioned meta info snapshot Key: FLINK-9512 URL: https://issues.apache.org/jira/browse/FLINK-9512 Project:

[jira] [Created] (FLINK-9511) Make StateDescriptor configurable with optional TTL

2018-06-04 Thread Andrey Zagrebin (JIRA)
Andrey Zagrebin created FLINK-9511: -- Summary: Make StateDescriptor configurable with optional TTL Key: FLINK-9511 URL: https://issues.apache.org/jira/browse/FLINK-9511 Project: Flink Issue

[jira] [Created] (FLINK-9510) State TTL API with cleanup on explicit access

2018-06-04 Thread Andrey Zagrebin (JIRA)
Andrey Zagrebin created FLINK-9510: -- Summary: State TTL API with cleanup on explicit access Key: FLINK-9510 URL: https://issues.apache.org/jira/browse/FLINK-9510 Project: Flink Issue Type:

Re: [Discuss] Proposing FLIP-25 - Support User State TTL Natively in Flink

2018-06-04 Thread Andrey Zagrebin
Hi everybody, We have been recently brainstorming ideas around state TTL in Flink and compiled our thoughts in the following design doc: https://docs.google.com/document/d/1SI_WoXAfOd4_NKpGyk4yh3mf59g12pSGNXRtNFi-tgM