Re: How to ensure that job is restored from savepoint when using Flink SQL

2020-07-08 Thread shadowell
Hi Fabian, Thanks for your reply, it helps a lot. Best Regards, Jie | | shadowell | | shadow...@126.com | 签名由网易邮箱大师定制 On 7/8/2020 18:17,Fabian Hueske wrote: Hi Jie, The auto-ID generation is not done by the SQL translation component but on a lower level, i.e., it's independent of

Re: How to ensure that job is restored from savepoint when using Flink SQL

2020-07-08 Thread Fabian Hueske
Hi Jie, The auto-ID generation is not done by the SQL translation component but on a lower level, i.e., it's independent of Flink's SQL translation. The ID generation only depends on the topology / graph structure of the program's operators. The ID of an operator depends on the IDs of its predeces

Re: How to ensure that job is restored from savepoint when using Flink SQL

2020-07-07 Thread shadowell
Hi Fabian, Thanks for your information! Actually, I am not clear about the mechanism of auto-generated IDs in Flink SQL and the mechanism of how does the operator state mapping back from savepoint. I hope to get some detail information by giving an example bellow. I have two sql as samples:

Re: How to ensure that job is restored from savepoint when using Flink SQL

2020-07-07 Thread Fabian Hueske
Hi Jie Feng, As you said, Flink translates SQL queries into streaming programs with auto-generated operator IDs. In order to start a SQL query from a savepoint, the operator IDs in the savepoint must match the IDs in the newly translated program. Right now this can only be guaranteed if you transl

How to ensure that job is restored from savepoint when using Flink SQL

2020-07-06 Thread shadowell
Hello, everyone, I have some unclear points when using Flink SQL. I hope to get an answer or tell me where I can find the answer. When using the DataStream API, in order to ensure that the job can recover the state from savepoint after adjustment, it is necessary to specify the