RE: Re: Re: Re: 日志里提示 The state is cleared because of state ttl. This will result in incorrect result 如何解决？

2022-04-20 文章 段晓雄 
我使用 ZooKeeper High-Availability with Standalone 集群模式，使用 sql-client.sh 提交任务

On 2022/04/20 10:43:16 Zhanghao Chen wrote:
> 你是用的什么 Flink 集群部署模式提交的 SQL 任务呢？
> 
> Best,
> Zhanghao Chen
> 
> From: 段晓雄 
> Sent: Saturday, April 16, 2022 19:52
> To: user-zh@flink.apache.org 
> Subject: RE: Re: Re: 日志里提示 The state is cleared because of state ttl. This 
> will result in incorrect result 如何解决？
> 
> Zhanghao,
> 
> 感谢帮助！我在 sql-client.sh 里设置了 table.exec.state.ttl= 
> 12960，但现在我不知道如何确认是否设置成功，我从web UI 和 rest api 返回的任务状态和checkpoint状态中都没有找到 
> state ttl 的值，我发现任务状态还是不断增大，我如何能确认任务 state ttl 的情况呢？
> 
> 
> On 2022/04/12 04:15:37 Zhanghao Chen wrote:
> > 你可以用 SQL client SET 'table.exec.state.ttl' = 'xxx'; 的语法来实现哈，具体可以参考下：
> >
> >
> >   1.  
> > https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sqlclient/#running-sql-queries
> > SQL Client | Apache 
> > Flink
> > The SET command allows you to tune the job execution and the sql client 
> > behaviour. See SQL Client Configuration below for more details.. After a 
> > query is defined, it can be submitted to the cluster as a long-running, 
> > detached Flink job. The configuration section explains how to declare table 
> > sources for reading data, how to declare table sinks for writing data, and 
> > how to configure other table ...
> > nightlies.apache.org
> >   2.
> >   3.  
> > https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/config/#table-exec-state-ttl
> > Configuration | Apache 
> > Flink
> > Configuration # By default, the Table & SQL API is preconfigured for 
> > producing accurate results with acceptable performance. Depending on the 
> > requirements of a table program, it might be necessary to adjust certain 
> > parameters for optimization. For example, unbounded streaming programs may 
> > need to ensure that the required state size is capped (see streaming 
> > concepts).
> > nightlies.apache.org
> >
> >
> > Best,
> > Zhanghao Chen
> > 
> > From: 段晓雄 
> > Sent: Monday, April 11, 2022 20:23
> > To: user-zh@flink.apache.org 
> > Subject: RE: Re: 日志里提示 The state is cleared because of state ttl. This will 
> > result in incorrect result 如何解决？
> >
> > 我使用sql-client.sh 执行sql创建的任务，如何设置 ttl 的时长？
> >
> > On 2022/04/11 11:14:36 yidan zhao wrote:
> > > You can increase the state ttl to avoid this.
> > > 这个已经明确了方法了，增加ttl的时长。
> > >
> > > 段晓雄  于2022年4月11日周一 09:52写道：
> > > >
> > > > 各位老大，
> > > >
> > > > 现在是 Flink 1.14.4 集群，通过 pyflink执行 sql 做流处理，
> > > >
> > > > taskmanager日志大量 The state is cleared because of state ttl. This will 
> > > > result in incorrect result. You can increase the state ttl to avoid 
> > > > this. 为什么出现？如何解决？
> > > >
> > > >
> > > > 2022-04-09 17:08:54,672 INFO  
> > > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > > >  [] - Committing the state for checkpoint 284
> > > > 2022-04-09 17:08:54,672 INFO  
> > > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > > >  [] - Committing the state for checkpoint 284
> > > > 2022-04-09 17:08:54,852 INFO  
> > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] 
> > > > - The state is cleared because of state ttl. This will result in 
> > > > incorrect result. You can increase the state ttl to avoid this.
> > > > 2022-04-09 17:08:54,852 INFO  
> > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] 
> > > > - The state is cleared because of state ttl. This will result in 
> > > > incorrect result. You can increase the state ttl to avoid this.
> > > > 2022-04-09 17:08:54,852 INFO  
> > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] 
> > > > - The state is cleared because of state ttl. This will result in 
> > > > incorrect result. You can increase the state ttl to avoid this.
> > > > 2022-04-09 17:08:54,922 INFO  
> > > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > > >  [] - Committing the state for checkpoint 277
> > > > 2022-04-09 17:08:54,922 INFO  
> > > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > > >  [] - Committing the state for checkpoint 277
> > > > 2022-04-09 17:08:54,952 INFO  
> > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] 
> > > > - The state is cleared because of state ttl. This will result in 
> > > > incorrect result. You can increase the state ttl to avoid this.
> > > > 2022-04-09 17:08:54,952 INFO  
> > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] 
> > > > - The state is cleared because of state ttl. This will result in 
> > > > incorrect result. You can increase the state

Re: Re: Re: 日志里提示 The state is cleared because of state ttl. This will result in incorrect result 如何解决？

2022-04-20 文章 Zhanghao Chen 
你是用的什么 Flink 集群部署模式提交的 SQL 任务呢？

Best,
Zhanghao Chen

From: 段晓雄 
Sent: Saturday, April 16, 2022 19:52
To: user-zh@flink.apache.org 
Subject: RE: Re: Re: 日志里提示 The state is cleared because of state ttl. This will 
result in incorrect result 如何解决？

Zhanghao,

感谢帮助！我在 sql-client.sh 里设置了 table.exec.state.ttl= 
12960，但现在我不知道如何确认是否设置成功，我从web UI 和 rest api 返回的任务状态和checkpoint状态中都没有找到 
state ttl 的值，我发现任务状态还是不断增大，我如何能确认任务 state ttl 的情况呢？


On 2022/04/12 04:15:37 Zhanghao Chen wrote:
> 你可以用 SQL client SET 'table.exec.state.ttl' = 'xxx'; 的语法来实现哈，具体可以参考下：
>
>
>   1.  
> https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sqlclient/#running-sql-queries
> SQL Client | Apache 
> Flink
> The SET command allows you to tune the job execution and the sql client 
> behaviour. See SQL Client Configuration below for more details.. After a 
> query is defined, it can be submitted to the cluster as a long-running, 
> detached Flink job. The configuration section explains how to declare table 
> sources for reading data, how to declare table sinks for writing data, and 
> how to configure other table ...
> nightlies.apache.org
>   2.
>   3.  
> https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/config/#table-exec-state-ttl
> Configuration | Apache 
> Flink
> Configuration # By default, the Table & SQL API is preconfigured for 
> producing accurate results with acceptable performance. Depending on the 
> requirements of a table program, it might be necessary to adjust certain 
> parameters for optimization. For example, unbounded streaming programs may 
> need to ensure that the required state size is capped (see streaming 
> concepts).
> nightlies.apache.org
>
>
> Best,
> Zhanghao Chen
> 
> From: 段晓雄 
> Sent: Monday, April 11, 2022 20:23
> To: user-zh@flink.apache.org 
> Subject: RE: Re: 日志里提示 The state is cleared because of state ttl. This will 
> result in incorrect result 如何解决？
>
> 我使用sql-client.sh 执行sql创建的任务，如何设置 ttl 的时长？
>
> On 2022/04/11 11:14:36 yidan zhao wrote:
> > You can increase the state ttl to avoid this.
> > 这个已经明确了方法了，增加ttl的时长。
> >
> > 段晓雄  于2022年4月11日周一 09:52写道：
> > >
> > > 各位老大，
> > >
> > > 现在是 Flink 1.14.4 集群，通过 pyflink执行 sql 做流处理，
> > >
> > > taskmanager日志大量 The state is cleared because of state ttl. This will 
> > > result in incorrect result. You can increase the state ttl to avoid this. 
> > > 为什么出现？如何解决？
> > >
> > >
> > > 2022-04-09 17:08:54,672 INFO  
> > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > >  [] - Committing the state for checkpoint 284
> > > 2022-04-09 17:08:54,672 INFO  
> > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > >  [] - Committing the state for checkpoint 284
> > > 2022-04-09 17:08:54,852 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,852 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,852 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,922 INFO  
> > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > >  [] - Committing the state for checkpoint 277
> > > 2022-04-09 17:08:54,922 INFO  
> > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > >  [] - Committing the state for checkpoint 277
> > > 2022-04-09 17:08:54,952 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,952 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,952 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,952 INFO  
> > >

RE: Re: Re: 日志里提示 The state is cleared because of state ttl. This will result in incorrect result 如何解决？

2022-04-17 文章 段晓雄 
Zhanghao,

感谢帮助！我在 sql-client.sh 里设置了 table.exec.state.ttl= 
12960，但现在我不知道如何确认是否设置成功，我从web UI 和 rest api 返回的任务状态和checkpoint状态中都没有找到 
state ttl 的值，我发现任务状态还是不断增大，我如何能确认任务 state ttl 的情况呢？


On 2022/04/12 04:15:37 Zhanghao Chen wrote:
> 你可以用 SQL client SET 'table.exec.state.ttl' = 'xxx'; 的语法来实现哈，具体可以参考下：
> 
> 
>   1.  
> https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sqlclient/#running-sql-queries
> SQL Client | Apache 
> Flink
> The SET command allows you to tune the job execution and the sql client 
> behaviour. See SQL Client Configuration below for more details.. After a 
> query is defined, it can be submitted to the cluster as a long-running, 
> detached Flink job. The configuration section explains how to declare table 
> sources for reading data, how to declare table sinks for writing data, and 
> how to configure other table ...
> nightlies.apache.org
>   2.
>   3.  
> https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/config/#table-exec-state-ttl
> Configuration | Apache 
> Flink
> Configuration # By default, the Table & SQL API is preconfigured for 
> producing accurate results with acceptable performance. Depending on the 
> requirements of a table program, it might be necessary to adjust certain 
> parameters for optimization. For example, unbounded streaming programs may 
> need to ensure that the required state size is capped (see streaming 
> concepts).
> nightlies.apache.org
> 
> 
> Best,
> Zhanghao Chen
> 
> From: 段晓雄 
> Sent: Monday, April 11, 2022 20:23
> To: user-zh@flink.apache.org 
> Subject: RE: Re: 日志里提示 The state is cleared because of state ttl. This will 
> result in incorrect result 如何解决？
> 
> 我使用sql-client.sh 执行sql创建的任务，如何设置 ttl 的时长？
> 
> On 2022/04/11 11:14:36 yidan zhao wrote:
> > You can increase the state ttl to avoid this.
> > 这个已经明确了方法了，增加ttl的时长。
> >
> > 段晓雄  于2022年4月11日周一 09:52写道：
> > >
> > > 各位老大，
> > >
> > > 现在是 Flink 1.14.4 集群，通过 pyflink执行 sql 做流处理，
> > >
> > > taskmanager日志大量 The state is cleared because of state ttl. This will 
> > > result in incorrect result. You can increase the state ttl to avoid this. 
> > > 为什么出现？如何解决？
> > >
> > >
> > > 2022-04-09 17:08:54,672 INFO  
> > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > >  [] - Committing the state for checkpoint 284
> > > 2022-04-09 17:08:54,672 INFO  
> > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > >  [] - Committing the state for checkpoint 284
> > > 2022-04-09 17:08:54,852 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,852 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,852 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,922 INFO  
> > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > >  [] - Committing the state for checkpoint 277
> > > 2022-04-09 17:08:54,922 INFO  
> > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler
> > >  [] - Committing the state for checkpoint 277
> > > 2022-04-09 17:08:54,952 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,952 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,952 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,952 INFO  
> > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] - 
> > > The state is cleared because of state ttl. This will result in incorrect 
> > > result. You can increase the state ttl to avoid this.
> > > 2022-04-09 17:08:54,952 INFO  
> > >