Re:Flink 截止到1.18,是否有办法在Table API上添加uid?

2024-04-24 Thread Xuyang
Hi, 如果在中间添加了op,或者修改了处理逻辑,那么代表拓扑图会变,那么基于拓扑序所确定的uid也会变,从状态恢复就可能失败。具体可以参考[1] 目前table api应该是没有开放自定义uid的能力,可以在jira[2]上新建一个feature的jira,然后在dev邮件里发起讨论下。 [1]

Flink 截止到1.18,是否有办法在Table API上添加uid?

2024-04-24 Thread Guanlin Zhang
Hi Team, 我们这边的业务使用 Flink MySQL CDC到 OpenSearch并且使用TABLE API: INSERT INTO t1 SELECT * FROM t2 这种方式。 由于我们这边可能会在运行过程中添加额外的Operator,我们有办法在使用snapshot 恢复后保留之前src和sink operator的状态么?我看到在DataStream API可以通过设定uid。Table API有同样的方法吗?我看到Flink jira:https://issues.apache.org/jira/browse/FLINK-28861

PyFlink Performance

2024-04-24 Thread David Jost
Hi, I am currently evaluating PyFlink in comparison to Java and did some various tests, mainly comparing identical pipelines with focus on throughput. For me it seems, that PyFlink is generally worse for wear and seems to reach its limits in throughput at a point where Java still has resources

Regarding java.lang.IllegalStateException

2024-04-24 Thread prashant parbhane
Hello, We have been facing this weird issue, where we are getting below exception and the job is getting restarted with new task managers. We are using flink 1.17. Same job works fine with lower number of task managers. (<10) java.lang.IllegalStateException: The library registration references a