Re: 退订

2023-02-23 Thread Jane Chan
Please refer to https://flink.apache.org/community/ and send the email to user-unsubscr...@flink.apache.org Best, Jane On Fri, Feb 24, 2023 at 3:17 PM zhangjunjie wrote: > 退订 > > >

退订

2023-02-23 Thread zhangjunjie
退订

RE: Re: Should we always mark ValueState as "transient" for RichFunctions

2023-02-23 Thread Zhongpu Chen
Hi Gen, Thanks for your explanation. Back to this code snippet, since they are not marked with "transient" now, I suppose Flink will use avro to serialize them (null values). Is there any benchmark to show the performance test between null values serialization and "transient"? I mean, it is

Re: Flink K8s operator pod section of CRD

2023-02-23 Thread Gyula Fóra
Hey! You are right, these fields could have been of the PodTemplate / PodTemplateSpec type (probably PodTemplateSpec is actually better). I think the reason why we used it is two fold: - Simple oversight :) - Flink itself "expects" the podtemplate in this form for the native integration as you

Re: Should we always mark ValueState as "transient" for RichFunctions

2023-02-23 Thread Gen Luo
Hi, ValueState is a handle rather than an actual value. So it should never be serialized. In fact, ValueState itself is not a Serializable. It should be ok to always mark it as transient. In this case, I suppose it works because the ValueState is not set (which happens during the runtime) when

使用flink sql 将kafka的数据同步到mysql无法删除。

2023-02-23 Thread 陈佳豪
-建表语法如下 String kafka = "CREATE TABLE `电话` " + "(`rowID` VARCHAR(255),`名称` STRING,`手机` VARCHAR(255),`座机` VARCHAR(255), " + " PRIMARY KEY (`rowID`) NOT ENFORCED ) " + " WITH " + "('connector' = 'jdbc', " + " 'driver' = 'com.mysql.cj.jdbc.Driver', " + " 'url' =

Re: flink taskmanger重启失败的问题

2023-02-23 Thread Weihua Hu
从 region 改为 full 会扩容单个 Task 故障的影响范围,可以参考社区文档: https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/ops/state/task_failure_recovery/ Best, Weihua On Fri, Feb 24, 2023 at 2:12 PM 唐世伟 wrote: > 谢谢回复,我看日志已经超出来yarn保存的期限被删了。另外Failover从region改为full。是不是能避免这个问题啊? > > > 2023年2月23日

Re: flink taskmanger重启失败的问题

2023-02-23 Thread 唐世伟
谢谢回复,我看日志已经超出来yarn保存的期限被删了。另外Failover从region改为full。是不是能避免这个问题啊? > 2023年2月23日 上午11:36,Weihua Hu 写道: > > Hi, > > 在 Cancel 其他 task 时会先将 task 状态置为 cancelling,这时 task 失败是不会二次触发 Failover 的。 > 可以检查下是不是作业划分了多个 region,多个 region 的异常是统一计数的。 > > 或者可以贴一下日志吗? > > > Best, > Weihua > > > On Thu, Feb 23,

Re: Re: Re: [急] flink sql写数据到带主键表时能确保相同主键的记录都被同一个taskmanager处理吗?

2023-02-23 Thread Shuo Cheng
> 你说的这个参数我看了默认值不是auto吗?需要我显式地指定为force? Sink upsert materialize would be applied in the following circumstances: 1. `TABLE_EXEC_SINK_UPSERT_MATERIALIZE` set to FORCE and sink's primary key nonempty. 2. `TABLE_EXEC_SINK_UPSERT_MATERIALIZE` set to AUTO and sink's primary key doesn't contain upsert

Should we always mark ValueState as "transient" for RichFunctions

2023-02-23 Thread Zhongpu Chen
Hi, When I am reading the code from flink-training-repo [1], I noticed the following code: ```java public static class EnrichmentFunction extends RichCoFlatMapFunction { private ValueStaterideState; private ValueStatefareState; ... } ``` From my understanding, since

Re:Re: Re: [急] flink sql写数据到带主键表时能确保相同主键的记录都被同一个taskmanager处理吗?

2023-02-23 Thread casel.chen
你说的这个参数我看了默认值不是auto吗?需要我显式地指定为force? Because of the disorder of ChangeLog data caused by Shuffle in distributed system, the data received by Sink may not be the order of global upsert. So add upsert materialize operator before upsert sink. It receives the upstream changelog records and

Flink K8s operator pod section of CRD

2023-02-23 Thread Mason Chen
Hi all, Why does the FlinkDeployment CRD refer to the Pod class instead of the PodTemplate class from the fabric8 library? As far as I can tell, the only difference is that the Pod class exposes the PodStatus, which doesn't seem mutable. Thanks in advance! Best, Mason

答复:

2023-02-23 Thread 704669594
退订

[Flink kubernetes operator] Usage of HorizontalPodAutoscaler

2023-02-23 Thread Ritesh Nadhani
Hello I was reviewing the HPA example at: https://github.com/apache/flink-kubernetes-operator/blob/main/examples/hpa/basic-hpa.yaml and I am a bit confused as to which spec this applies to. Practically, I would assume that it's for scaling TaskManager worker pods. Is that correct? Then how does

[ANNOUNCE] Apache Flink Kubernetes Operator 1.4.0 released

2023-02-23 Thread Gyula Fóra
The Apache Flink community is very happy to announce the release of Apache Flink Kubernetes Operator 1.4.0. The Flink Kubernetes Operator allows users to manage their Apache Flink applications and their lifecycle through native k8s tooling like kubectl. Release highlights: - Flink Job

[ANNOUNCE] Apache Flink Kubernetes Operator 1.4.0 released

2023-02-23 Thread Gyula Fóra
The Apache Flink community is very happy to announce the release of Apache Flink Kubernetes Operator 1.4.0. The Flink Kubernetes Operator allows users to manage their Apache Flink applications and their lifecycle through native k8s tooling like kubectl. Release highlights: - Flink Job

Re: Kubernetes operator's merging strategy for template arrays

2023-02-23 Thread Alexis Sarda-Espinosa
Ah I see, I'll have a look, thanks. Am Do., 23. Feb. 2023 um 14:21 Uhr schrieb Gyula Fóra : > If you are interested in helping to review this, here is the relevant > ticket and the PR I just opened: > > https://issues.apache.org/jira/browse/FLINK-30786 >

Re: Kubernetes operator's merging strategy for template arrays

2023-02-23 Thread Gyula Fóra
If you are interested in helping to review this, here is the relevant ticket and the PR I just opened: https://issues.apache.org/jira/browse/FLINK-30786 https://github.com/apache/flink-kubernetes-operator/pull/535 Cheers, Gyula On Thu, Feb 23, 2023 at 2:10 PM Gyula Fóra wrote: > Hi! > > The

Re: Kubernetes operator's merging strategy for template arrays

2023-02-23 Thread Gyula Fóra
Hi! The current array merging strategy in the operator is basically an overwrite by position yes. I actually have a pending improvement to make this configurable and allow merging arrays by "name" attribute. This is generally more practical for such cases. Cheers, Gyula On Thu, Feb 23, 2023 at

Kubernetes operator's merging strategy for template arrays

2023-02-23 Thread Alexis Sarda-Espinosa
Hello, I noticed that if I set environment variables in both spec.podTemplate & spec.jobManager.podTemplate for the same container (flink-maincontainer), the values from the latter selectively overwrite the values from the former. For example, if I define something like this (omitting metadata