Re: Support for multiple HDFS

2017-08-23 Thread Vijay Srinivasaraghavan
Hi Ted, I believe HDFS-6584 is more of an HDFS feature supporting archive use case through some policy configurations. My ask is that I have two distinct HCFS File systems which are independent but the Flink job will decide which one to use for sink while the Flink infrastructure is by default

Re: Support for multiple HDFS

2017-08-23 Thread Ted Yu
Would HDFS-6584 help with your use case ? On Wed, Aug 23, 2017 at 11:00 AM, Vijay Srinivasaraghavan < vijikar...@yahoo.com.invalid> wrote: > Hello, > Is it possible for a Flink cluster to use multiple HDFS repository (HDFS-1 > for managing Flink state backend, HDFS-2 for syncing results from

Support for multiple HDFS

2017-08-23 Thread Vijay Srinivasaraghavan
Hello, Is it possible for a Flink cluster to use multiple HDFS repository (HDFS-1 for managing Flink state backend, HDFS-2 for syncing results from user job)?  The scenario can be viewed in the context of running some jobs that are meant to push the results to an archive repository (cold

[jira] [Created] (FLINK-7495) AbstractUdfStreamOperator#initializeState() should be called in AsyncWaitOperator#initializeState()

2017-08-23 Thread Ted Yu (JIRA)
Ted Yu created FLINK-7495: - Summary: AbstractUdfStreamOperator#initializeState() should be called in AsyncWaitOperator#initializeState() Key: FLINK-7495 URL: https://issues.apache.org/jira/browse/FLINK-7495

[jira] [Created] (FLINK-7494) No license headers in ".travis.yml" file

2017-08-23 Thread Hai Zhou (JIRA)
Hai Zhou created FLINK-7494: --- Summary: No license headers in ".travis.yml" file Key: FLINK-7494 URL: https://issues.apache.org/jira/browse/FLINK-7494 Project: Flink Issue Type: Wish

Re: [DISCUSS] Flink 1.4 and time based release

2017-08-23 Thread Timo Walther
I also think we shouldn't publish releases regularly, just to have a release regularly. Maybe we can do time-based releases more flexible: Instead of feature-freeze after 3 months, 1 month testing. We could do it like feature-freeze 3 months after the last release, unlimited testing. This

Re: [DISCUSS] Stop serving docs for Flink version prior to 1.0

2017-08-23 Thread Ufuk Celebi
Thanks Timo for taking this over :-) On Wed, Aug 23, 2017 at 10:51 AM, Timo Walther wrote: > I had a offline discussion with Ufuk. I looked into the docs build scripts > recently, I can take care of removing the old docs. There other issues that > need to be fixed before the

Re: [DISCUSS] Stop serving docs for Flink version prior to 1.0

2017-08-23 Thread Timo Walther
I had a offline discussion with Ufuk. I looked into the docs build scripts recently, I can take care of removing the old docs. There other issues that need to be fixed before the next release as well: - Drop docs < 1.0 - Make Javadocs with Scala build again - Build all docs >= 1.0 again (esp.

Re: [DISCUSS] Flink 1.4 and time based release

2017-08-23 Thread Till Rohrmann
Thanks for starting the discussion Stephan. I agree with you that the last release was probably a bit hasty due to the constraints we put on ourselves with the strict time based release. Therefore and because of some of the incomplete features, I would be in favour of loosening the strict deadline

Re: [DISCUSS] Stop serving docs for Flink version prior to 1.0

2017-08-23 Thread Piotr Nowojski
+1 Is there a way to serve “current” docs under some permanent link, showing docs from whatever is the latest release? Maybe such docs could be indexed by the google and ranked higher then any particular release mitigating this issue for the future? > On Aug 22, 2017, at 6:48 PM, Till