Re: [DISCUSS] Integrate Flink SQL well with Hive ecosystem

2018-11-17 Thread Xiaowei Jiang
Thanks Xuefu for the detailed design doc! One question on the properties associated with the catalog objects. Are we going to leave them completely free form or we are going to set some standard for that? I think that the answer may depend on if we want to explore catalog specific optimization

Re: [DISCUSS] Task speculative execution for Flink batch

2018-11-17 Thread Xiaowei Jiang
Thanks Yangyu for the nice design doc! One thing to consider is the granularity of speculation. Multiple task may propagate data through pipeline mode. In such case, fixing a single task may not be enough. But you might be able to fix this problem by increasing the granularity of speculation. The

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2018-11-17 Thread Thomas Weise
@Aljoscha to address your question first: In the case of the Kinesis consumer (with current Kinesis consumer API), there would also be N+1 threads. I have implemented a prototype similar to what is shown in Jamie's document, where the thread ownership is similar to what you have done for Kafka.

Re: [DISCUSS] Task speculative execution for Flink batch

2018-11-17 Thread Tao Yangyu
Hi all, After refined, the detailed design doc is here: https://docs.google.com/document/d/1X_Pfo4WcO-TEZmmVTTYNn44LQg5gnFeeaeqM7ZNLQ7M/edit?usp=sharing Your kind reviews and comments are very appreciated and will help so much the feature to be completed. Best, Ryan Tao Yangyu 于2018年11月7日周三

[jira] [Created] (FLINK-10915) clojure context.collectWithTimestamp Will be blocked.

2018-11-17 Thread fengge (JIRA)
fengge created FLINK-10915: -- Summary: clojure context.collectWithTimestamp Will be blocked. Key: FLINK-10915 URL: https://issues.apache.org/jira/browse/FLINK-10915 Project: Flink Issue Type:

[jira] [Created] (FLINK-10914) Prefer notifying status changed to waiting and checking in ExecutionGraph tests

2018-11-17 Thread TisonKun (JIRA)
TisonKun created FLINK-10914: Summary: Prefer notifying status changed to waiting and checking in ExecutionGraph tests Key: FLINK-10914 URL: https://issues.apache.org/jira/browse/FLINK-10914 Project:

[jira] [Created] (FLINK-10913) ExecutionGraphRestartTest.testRestartAutomatically unstable on Travis

2018-11-17 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-10913: - Summary: ExecutionGraphRestartTest.testRestartAutomatically unstable on Travis Key: FLINK-10913 URL: https://issues.apache.org/jira/browse/FLINK-10913 Project: