[jira] [Created] (FLINK-10168) support filtering files by modified/created time in StreamExecutionEnvironment.readFile()

2018-08-17 Thread Bowen Li (JIRA)
Bowen Li created FLINK-10168: Summary: support filtering files by modified/created time in StreamExecutionEnvironment.readFile() Key: FLINK-10168 URL: https://issues.apache.org/jira/browse/FLINK-10168

Re: [Proposal] Utilities for reading, transforming and creatingStreaming savepoints

2018-08-17 Thread 陈梓立
Hi, also +1. As vivo said, SavePoint is not compatible. I have heard from a lot of users just said "my previous program does not work any more!". If these utilities provide such migration functions, it would be perfect! Best, tison. vino yang 于2018年8月18日周六 上午11:04写道: > Hi, > > +1, from my

Re: [Proposal] Utilities for reading, transforming and creatingStreaming savepoints

2018-08-17 Thread vino yang
Hi, +1, from my side. Considering that Savepoint is not compatible with every version of Flink. It would be very useful if it could do some conversion between different versions of Flink Savepoint. Thanks, vino. bupt_ljy 于2018年8月18日周六 上午2:55写道: > Hi, > +1, I think it will be a very great tool

[jira] [Created] (FLINK-10167) SessionWindows not compatible with typed DataStreams in scala

2018-08-17 Thread Andrew Roberts (JIRA)
Andrew Roberts created FLINK-10167: -- Summary: SessionWindows not compatible with typed DataStreams in scala Key: FLINK-10167 URL: https://issues.apache.org/jira/browse/FLINK-10167 Project: Flink

Re: [Proposal] Utilities for reading, transforming and creatingStreaming savepoints

2018-08-17 Thread bupt_ljy
Hi, +1, I think it will be a very great tool for Flink, especially the creating new state part. On production, we’re really worried about the availability of the savepoints, because the generating logic is inside Flink and we don’t have a good way to validate it. But with this tool, we can

Re: [Proposal] Utilities for reading, transforming and creating Streaming savepoints

2018-08-17 Thread Jamie Grier
This is great, Gyula! A colleague here at Lyft has also done some work around bootstrapping DataStream programs and we've also talked a bit about doing this by running DataSet programs. On Fri, Aug 17, 2018 at 3:28 AM, Gyula Fóra wrote: > Hi All! > > I want to share with you a little project

Re: [VOTE] Release 1.5.3, release candidate #1

2018-08-17 Thread Till Rohrmann
+1 (binding) - built Flink from source release with Hadoop version 2.8.4 - executed all end-to-end tests sucessfully - executed Jepsen test suite successfully with binary release Cheers, Till On Thu, Aug 16, 2018, 12:59 Chesnay Schepler wrote: > Hi everyone, > Please review and vote on the

Re: [Proposal] Utilities for reading, transforming and creating Streaming savepoints

2018-08-17 Thread Gyula Fóra
Thanks for the feedback :) I agree that combining this with SQL would give an extremely nice layer to analyse the states. Our goal is to contribute this to Flink, I think this should live as part of the Flink project to make deeper intergration possible in the long run. Of course a pre-requisite

Re: [Proposal] Utilities for reading, transforming and creating Streaming savepoints

2018-08-17 Thread Piotr Nowojski
Hi, Very huge +1 from my side. I found lack of such tool/possibility as a big problem for long term maintainability of Flink jobs. In the long run, I would be delight to see Flink SQL support for those things as well. Ad hoc analysis is one of the prime use case of SQL. This tool would make

Re: How to handle tests that need docker?

2018-08-17 Thread Gary Yao
Hi, If you are writing JUnit tests, you could use testcontainers [1] to start the PubSub container. Then you could introduce a build profile [2] that enables certain JUnit tests by using Categories [3] or the maven-failsafe-plugin [4]. Best, Gary [1]

[Proposal] Utilities for reading, transforming and creating Streaming savepoints

2018-08-17 Thread Gyula Fóra
Hi All! I want to share with you a little project we have been working on at King (with some help from some dataArtisans folks). I think this would be a valuable addition to Flink and solve a bunch of outstanding production use-cases and headaches around state bootstrapping and state analytics.

[jira] [Created] (FLINK-10166) Dependency problems when executing SQL query in sql-client

2018-08-17 Thread Dawid Wysakowicz (JIRA)
Dawid Wysakowicz created FLINK-10166: Summary: Dependency problems when executing SQL query in sql-client Key: FLINK-10166 URL: https://issues.apache.org/jira/browse/FLINK-10166 Project: Flink

[jira] [Created] (FLINK-10165) JarRunHandler/JarRunRequestBody should allow to pass program arguments as escaped json list

2018-08-17 Thread Maciej Prochniak (JIRA)
Maciej Prochniak created FLINK-10165: Summary: JarRunHandler/JarRunRequestBody should allow to pass program arguments as escaped json list Key: FLINK-10165 URL: