Re: [DISCUSS] FLIP-132: Temporal Table DDL

2020-08-20 Thread Jark Wu
Hi everyone, Thank you for the great discussion @Leonard and @Fabian. *Regarding to choose a different name for this join:* >From my point of view, I don't agree to introduce a new grammar called whatever "lookup join" or "version join", because: 1. "lookup" is a physical behavior not a logical

[jira] [Created] (FLINK-19011) Parallelize the restoreOperation in OperatorStateBackend

2020-08-20 Thread Jiayi Liao (Jira)
Jiayi Liao created FLINK-19011: -- Summary: Parallelize the restoreOperation in OperatorStateBackend Key: FLINK-19011 URL: https://issues.apache.org/jira/browse/FLINK-19011 Project: Flink Issue

[jira] [Created] (FLINK-19010) Add a system metric to show the checkpoint restore time

2020-08-20 Thread Zhinan Cheng (Jira)
Zhinan Cheng created FLINK-19010: Summary: Add a system metric to show the checkpoint restore time Key: FLINK-19010 URL: https://issues.apache.org/jira/browse/FLINK-19010 Project: Flink

[jira] [Created] (FLINK-19009) wrong way to calculate the "downtime" metric

2020-08-20 Thread Zhinan Cheng (Jira)
Zhinan Cheng created FLINK-19009: Summary: wrong way to calculate the "downtime" metric Key: FLINK-19009 URL: https://issues.apache.org/jira/browse/FLINK-19009 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-19008) Flink Job runs slow after restore + downscale from an incremental checkpoint (rocksdb)

2020-08-20 Thread Jun Qin (Jira)
Jun Qin created FLINK-19008: --- Summary: Flink Job runs slow after restore + downscale from an incremental checkpoint (rocksdb) Key: FLINK-19008 URL: https://issues.apache.org/jira/browse/FLINK-19008

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

2020-08-20 Thread Matthias Pohl
Good points, Andrey. Thanks for clarification. I made some minor adaptations to the FLIP now: - Renamed the `resource` member into `configuration` and made it a top-level member besides `metrics` and `hardware` since it's not fitting the volatile metrics context that well. - I restructured the

[jira] [Created] (FLINK-19007) Automatically set presto staging directory to io.tmp.dirs

2020-08-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-19007: Summary: Automatically set presto staging directory to io.tmp.dirs Key: FLINK-19007 URL: https://issues.apache.org/jira/browse/FLINK-19007 Project: Flink

Re: Decompose failure recovery time

2020-08-20 Thread Zhinan Cheng
Hi Piotr, Thanks a lot. I will try your suggestion to see what happen. Regards, Zhinan On Fri, 21 Aug 2020 at 00:40, Piotr Nowojski wrote: > > Hi Zhinan, > > It's hard to say, but my guess it takes that long for the tasks to respond to > cancellation which consists of a couple of steps. If a

Re: Decompose failure recovery time

2020-08-20 Thread Piotr Nowojski
Hi Zhinan, It's hard to say, but my guess it takes that long for the tasks to respond to cancellation which consists of a couple of steps. If a task is currently busy processing something, it has to respond to interruption (`java.lang.Thread#interrupt`). If it takes 30 seconds for a task to react

Re: Decompose failure recovery time

2020-08-20 Thread Zhinan Cheng
Hi Piotr, Thanks a lot for your help. Yes, I finally realize that I can only approximate the time for [1] and [3] and measure [2] by monitoring the uptime and downtime metric provided by Flink. And now my problem is that I found the time in [2] can be up to 40s, I wonder why it takes so long to

Re: [DISCUSS] FLIP-132: Temporal Table DDL

2020-08-20 Thread Fabian Hueske
Hi everyone, Yes, just to help user distinguish the difference between versioned > temporal table and latest-only temporal table. > > I don't think we help users to understand the differences if we invent new (IMO confusing) terms ("temporal table without version" or "latest-only temporal

Re: [DISCUSS] FLIP-134: DataStream Semantics for Bounded Input

2020-08-20 Thread Kostas Kloudas
Hi all, Thanks for the comments! @Dawid: "execution.mode" can be a nice alternative and from a quick look it is not used currently by any configuration option. I will update the FLIP accordingly. @David: Given that having the option to allow timers to fire at the end of the job is already in

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

2020-08-20 Thread Andrey Zagrebin
Hi All, Thanks for reviving the discussion, Matthias! This would mean that we could adapt the current proposal to replace the > Nonheap usage pane by a pane displaying the Metaspace usage. > I do not know the value of having the Nonheap usage in metrics. I can see that the metaspace metric can

[RESULT] [VOTE] Release 1.10.2, release candidate #2

2020-08-20 Thread Zhu Zhu
Hi everyone, I'm happy to announce that we have unanimously approved this release. There are 6 approving votes, 3 of which are binding: * Robert Metzger (binding) * Till Rohrmann (binding) * Ufuk Celebi (binding) * Jeff Zhang * Congxian Qiu * Yun Tang There are no disapproving votes. Thanks

Re: Decompose failure recovery time

2020-08-20 Thread Piotr Nowojski
Hi, > I want to decompose the recovery time into different parts, say > (1) the time to detect the failure, > (2) the time to restart the job, > (3) and the time to restore the checkpointing. 1. Maybe I'm missing something, but as far as I can tell, Flink can not help you with that. Time to

[jira] [Created] (FLINK-19006) project transformation does not support conversion to Tuple25 type

2020-08-20 Thread ming li (Jira)
ming li created FLINK-19006: --- Summary: project transformation does not support conversion to Tuple25 type Key: FLINK-19006 URL: https://issues.apache.org/jira/browse/FLINK-19006 Project: Flink

Re: [VOTE] Release 1.10.2, release candidate #2

2020-08-20 Thread Yun Tang
Thanks for Zhu Zhu's quick response and LGTM now, +1 (non-binding) Best Yun Tang From: Zhu Zhu Sent: Thursday, August 20, 2020 19:39 To: dev ; Yun Tang Subject: Re: [VOTE] Release 1.10.2, release candidate #2 Thanks for pointing it out! @Yun

[jira] [Created] (FLINK-19005) used metaspace grow on every execution

2020-08-20 Thread Jira
Guillermo Sánchez created FLINK-19005: - Summary: used metaspace grow on every execution Key: FLINK-19005 URL: https://issues.apache.org/jira/browse/FLINK-19005 Project: Flink Issue Type:

Re: [VOTE] Release 1.10.2, release candidate #2

2020-08-20 Thread Zhu Zhu
Thanks for pointing it out! @Yun Tang The specific release notes of FLINK-18242 was missing. I have just added the notes for it in the pull request of release notes page (https://github.com/apache/flink-web/pull/366), as well as the other missing release notes of FLINK-17800. Thanks, Zhu Zhu

[jira] [Created] (FLINK-19004) Fail to call Hive percentile function together with distinct aggregate call

2020-08-20 Thread Rui Li (Jira)
Rui Li created FLINK-19004: -- Summary: Fail to call Hive percentile function together with distinct aggregate call Key: FLINK-19004 URL: https://issues.apache.org/jira/browse/FLINK-19004 Project: Flink

Re: [VOTE] Release 1.10.2, release candidate #2

2020-08-20 Thread Yun Tang
I checked to build from source code and run example on YARN and they all behave as expected. However, I think we should tell user explicitly for the interface changes to fix FLINK-18242 and already leave comments for the flink-web PR. Best Yun Tang From:

Re: [DISCUSS] Removing deprecated methods from DataStream API

2020-08-20 Thread Stephan Ewen
We have removed some public methods in the past, after a careful deprecation period, if they were not well working any more. The sentiment I got from users is that careful cleanup is in fact appreciated, otherwise things get confusing over time (the deprecated methods cause noise in the API).

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

2020-08-20 Thread Matthias
Hi Jing, I recently joined Ververica and started looking into FLIP-102. I'm trying to figure out how we would implement the proposal on the backend side. I looked into the proposal for the REST API response and a few questions popped up: - Is there a reason for us to introduce a nested structure

Re: [DISCUSS] Removing deprecated methods from DataStream API

2020-08-20 Thread Dawid Wysakowicz
Hey Till, You've got a good point here. Removing some of the methods would cause breaking the stability guarantees. I do understand if we decide not to remove them for that reason, let me explain though why I am thinking it might make sense to remove them already. First of all I am a bit afraid