rmj1218 opened a new issue, #9112: URL: https://github.com/apache/seatunnel/issues/9112
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened 部署了Zeta分离模式集群，1个master，3个worker。batch模式，从Elasticsearch同步数据到paimon。es索引69个字段，124w行数据，约1.3g。 同步效率最高不超过5000行/每秒。 top查看3个worker机器，发现每次只有一个worker在工作。怎么能通过调整配置及参数，提高同步效率呢？ ### SeaTunnel Version seatunnel-2.3.9 ### SeaTunnel Config ```conf hazelcast-master.yaml hazelcast: cluster-name: seatunnel network: rest-api: enabled: true endpoint-groups: CLUSTER_WRITE: enabled: true DATA: enabled: true join: tcp-ip: enabled: true member-list: - 10.1.5.xxx:5801 - 10.1.5.xxx:5802 - 10.1.5.xxx:5802 - 10.1.5.xxx:5802 port: auto-increment: false port: 5801 properties: hazelcast.invocation.max.retry.count: 20 hazelcast.tcp.join.port.try.count: 30 hazelcast.logging.type: log4j2 hazelcast.operation.generic.thread.count: 50 hazelcast.heartbeat.failuredetector.type: phi-accrual hazelcast.heartbeat.interval.seconds: 2 hazelcast.max.no.heartbeat.seconds: 180 hazelcast.heartbeat.phiaccrual.failuredetector.threshold: 10 hazelcast.heartbeat.phiaccrual.failuredetector.sample.size: 200 hazelcast.heartbeat.phiaccrual.failuredetector.min.std.dev.millis: 100 seatunnel.yaml seatunnel: engine: classloader-cache-mode: true history-job-expire-minutes: 1440 backup-count: 1 queue-type: blockingqueue print-execution-info-interval: 60 print-job-metrics-info-interval: 60 slot-service: dynamic-slot: false slot-num: 30 checkpoint: interval: 300000 timeout: 60000 storage: type: hdfs max-retained: 3 plugin-config: namespace: /tmp/seatunnel/checkpoint_snapshot storage.type: hdfs fs.defaultFS: file:///tmp/ # Ensure that the directory has written permission telemetry: metric: enabled: false log: scheduled-deletion-enable: true http: enable-http: true port: 8080 enable-dynamic-port: false coordinator-service: core-thread-num: 30 es2paimon_alert_test.conf env { parallelism = 5 job.mode = "BATCH" checkpoint.interval = 600000 checkpoint.timeout = 600000 } source { Elasticsearch { plugin_output = "fake" hosts = ["http://10.1.xxx.xx:9200";] index = "alert_${ptVal}*" query = { "match_all": {} } scroll_size=10000 source = ["_id","cancel_people","site_type","alert_start_time","cancel_remark","device_type","threshold","source_room","force_clear_reason","index_seq","order_status","biz_sys","mete_name","classify_type","alert_id","engineering_status","alert_explain","alert_level","order_type","status_masks","cancel_time","third_party_flag","create_time","device_id","task_flag","precinct_id","object_id","province_name","lsc_id","site_name","room_name","show_type","full_name","change_plan","rce_clear_alert_id","confirm_people","fixed","order_id","serial_no","room_id","room_kind_name","raw_mete_code","remark","alert_type","city_name","device_name","source_alert_id","sub_logical_type","alert_start_month","confirm_remark","device_type_name","mete_id","cur_moni_value","force_clear","object_type","lsc_name","raw_mete_name","r_alert_id","manufacturer_name","mete_code","confirm_state","confirm_time","alert_end_time","clear_value","is_standard_level","cur_moni_time","logical_type","namespace","alert_ reason","precinct_name"] } } transform { Sql { plugin_input = "fake" plugin_output = "fake1" query = "select _id as index_id, FORMATDATETIME(create_time,'yyyyMM') as pt,cancel_people,site_type,alert_start_time,cancel_remark,device_type,threshold,source_room,force_clear_reason,index_seq,order_status,biz_sys,mete_name,classify_type,alert_id,engineering_status,alert_explain,alert_level,order_type,status_masks,cancel_time,third_party_flag,create_time,device_id,task_flag,precinct_id,object_id,province_name,lsc_id,site_name,room_name,show_type,full_name,change_plan,rce_clear_alert_id,confirm_people,fixed,order_id,serial_no,room_id,room_kind_name,raw_mete_code,remark,alert_type,city_name,device_name,source_alert_id,sub_logical_type,alert_start_month,confirm_remark,device_type_name,mete_id,cur_moni_value,force_clear,object_type,lsc_name,raw_mete_name,r_alert_id,manufacturer_name,mete_code,confirm_state,confirm_time,alert_end_time,clear_value,is_standard_level,cur_moni_time,logical_type,namespace,alert_reason,precinct_name from dual" } } sink { Paimon { plugin_input = "fake1" catalog_name="es_catalog" warehouse="hdfs://NS1/es2paimon/" database="paimon_db" table="alert_pt_test" paimon.hadoop.conf = { security.kerberos.login.principal = "xxxx/admin@BCHKDC" security.kerberos.login.keytab = "/home/sudoroot/bdoc.keytab" } paimon.hadoop.conf-path="/opt/aspire/bigdata/seatunnel-2.3.9/config" } } ``` ### Running Command ```shell ./bin/seatunnel.sh --config config/es2paimon_alert_test.conf -i ptVal=202405 2025-04-03 11:17:35,096 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/aspire/bigdata/seatunnel-2.3.9/config/seatunnel.yaml' from System property 'seatunnel.config' 2025-04-03 11:17:35,100 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/aspire/bigdata/seatunnel-2.3.9/config/seatunnel.yaml 2025-04-03 11:17:35,102 INFO [o.a.s.e.c.c.SeaTunnelConfig ] [main] - seatunnel.home is /opt/aspire/bigdata/seatunnel-2.3.9 2025-04-03 11:17:35,211 WARN [amlSeaTunnelDomConfigProcessor] [main] - Unrecognized element: log 2025-04-03 11:17:35,212 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/aspire/bigdata/seatunnel-2.3.9/config/hazelcast.yaml' from System property 'hazelcast.config' 2025-04-03 11:17:35,212 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/aspire/bigdata/seatunnel-2.3.9/config/hazelcast.yaml 2025-04-03 11:17:35,581 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Loading configuration '/opt/aspire/bigdata/seatunnel-2.3.9/config/hazelcast-client.yaml' from System property 'hazelcast.client.config' 2025-04-03 11:17:35,581 INFO [c.h.i.c.AbstractConfigLocator ] [main] - Using configuration file at /opt/aspire/bigdata/seatunnel-2.3.9/config/hazelcast-client.yaml 2025-04-03 11:17:35,896 INFO [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel] [5.1] Running with 2 response threads, dynamic=true 2025-04-03 11:17:35,958 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING 2025-04-03 11:17:35,959 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED 2025-04-03 11:17:35,982 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to cluster: seatunnel 2025-04-03 11:17:35,985 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Trying to connect to [10.1.5.111]:5801 2025-04-03 11:17:36,021 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED 2025-04-03 11:17:36,021 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Authenticated with server [10.1.5.111]:5801:dc289e39-232e-44d7-b3c1-38f5d90b9337, server version: 5.1, local address: /10.1.5.111:53433 2025-04-03 11:17:36,023 INFO [c.h.i.d.Diagnostics ] [main] - hz.client_1 [seatunnel] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments. 2025-04-03 11:17:36,053 INFO [c.h.c.i.s.ClientClusterService] [hz.client_1.event-5] - hz.client_1 [seatunnel] [5.1] Members [4] { Member [10.1.5.111]:5801 - dc289e39-232e-44d7-b3c1-38f5d90b9337 [master node] Member [10.1.5.xxx]:5802 - 4ca6b7ba-6324-40f1-a213-8cdafe0515ff [worker node] Member [10.1.5.xxx]:5802 - a1c57590-7fa6-497a-9258-2130da470a34 [worker node] Member [10.1.5.xxx]:5802 - 31916a93-982b-4f47-aba6-72505b82bb81 [worker node] } 2025-04-03 11:17:36,061 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Authenticated with server [10.1.5.109]:5802:4ca6b7ba-6324-40f1-a213-8cdafe0515ff, server version: 5.1, local address: /10.1.5.111:53941 2025-04-03 11:17:36,401 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Authenticated with server [10.1.5.112]:5802:a1c57590-7fa6-497a-9258-2130da470a34, server version: 5.1, local address: /10.1.5.111:48085 2025-04-03 11:17:36,406 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel] [5.1] Authenticated with server [10.1.5.113]:5802:31916a93-982b-4f47-aba6-72505b82bb81, server version: 5.1, local address: /10.1.5.111:36281 2025-04-03 11:17:36,457 INFO [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds. 2025-04-03 11:17:37,686 INFO [o.a.s.c.s.u.ConfigBuilder ] [main] - Loading config file from path: config/es2paimon_alert_test.conf 2025-04-03 11:17:37,800 INFO [o.a.s.c.s.u.ConfigShadeUtils ] [main] - Load config shade spi: [base64] 2025-04-03 11:17:37,842 INFO [o.a.s.c.s.u.ConfigBuilder ] [main] - Parsed config file: { "env" : { "parallelism" : 5, "job.mode" : "BATCH", "checkpoint.interval" : 600000, "checkpoint.timeout" : 600000 }, "source" : [ { "plugin_output" : "fake", "hosts" : [ "http://10.1.xxx.xx:9200"; ], "index" : "alert_202405*", "query" : { "match_all" : {} }, "scroll_size" : 10000, "source" : [ "_id", "cancel_people", "site_type", "alert_start_time", "cancel_remark", "device_type", "threshold", "source_room", "force_clear_reason", "index_seq", "order_status", "biz_sys", "mete_name", "classify_type", "alert_id", "engineering_status", "alert_explain", "alert_level", "order_type", "status_masks", "cancel_time", "third_party_flag", "create_time", "device_id", "task_flag", "precinct_id", "object_id", "province_name", "lsc_id", "site_name", "room_name", "show_type", "full_name", "change_plan", "rce_clear_alert_id", "confirm_people", "fixed", "order_id", "serial_no", "room_id", "room_kind_name", "raw_mete_code", "remark", "alert_type", "city_name", "device_name", "source_alert_id", "sub_logical_type", "alert_start_month", "confirm_remark", "device_type_name", "mete_id", "cur_moni_value", "force_clear", "object_type", "lsc_name", "raw_mete_name", "r_alert_id", "manufacturer_name", "mete_code", "confirm_state", "confirm_time", "alert_end_time", "clear_value", "is_standard_level", "cur_moni_time", "logical_type", "namespace", "alert_reason", "precinct_name" ], "plugin_name" : "Elasticsearch" } ], "transform" : [ { "plugin_input" : "fake", "plugin_output" : "fake1", "query" : "select _id as index_id, FORMATDATETIME(create_time,'yyyyMM') as pt,cancel_people,site_type,alert_start_time,cancel_remark,device_type,threshold,source_room,force_clear_reason,index_seq,order_status,biz_sys,mete_name,classify_type,alert_id,engineering_status,alert_explain,alert_level,order_type,status_masks,cancel_time,third_party_flag,create_time,device_id,task_flag,precinct_id,object_id,province_name,lsc_id,site_name,room_name,show_type,full_name,change_plan,rce_clear_alert_id,confirm_people,fixed,order_id,serial_no,room_id,room_kind_name,raw_mete_code,remark,alert_type,city_name,device_name,source_alert_id,sub_logical_type,alert_start_month,confirm_remark,device_type_name,mete_id,cur_moni_value,force_clear,object_type,lsc_name,raw_mete_name,r_alert_id,manufacturer_name,mete_code,confirm_state,confirm_time,alert_end_time,clear_value,is_standard_level,cur_moni_time,logical_type,namespace,alert_reason,precinct_name from dual", "plugin_name" : "Sql" } ], "sink" : [ { "plugin_input" : "fake1", "catalog_name" : "es_catalog", "warehouse" : "hdfs://NS1/es2paimon/", "database" : "paimon_db", "table" : "alert_pt_test", "paimon.hadoop.conf" : { "security.kerberos.login.principal" : "xxxx/admin@BCHKDC", "security.kerberos.login.keytab" : "/home/sudoroot/bdoc.keytab" }, "paimon.hadoop.conf-path" : "/opt/aspire/bigdata/seatunnel-2.3.9/config", "plugin_name" : "Paimon" } ] } 2025-04-03 11:17:37,850 INFO [p.MultipleTableJobConfigParser] [main] - add common jar in plugins :[] 2025-04-03 11:17:37,858 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/aspire/bigdata/seatunnel-2.3.9/connectors 2025-04-03 11:17:37,862 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Elasticsearch'} at: file:/opt/aspire/bigdata/seatunnel-2.3.9/connectors/connector-elasticsearch-2.3.9.jar 2025-04-03 11:17:37,864 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/aspire/bigdata/seatunnel-2.3.9/connectors 2025-04-03 11:17:37,864 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='transform', pluginName='Sql'} at: file:/opt/aspire/bigdata/seatunnel-2.3.9/connectors/seatunnel-transforms-v2-2.3.9.jar 2025-04-03 11:17:37,867 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/aspire/bigdata/seatunnel-2.3.9/connectors 2025-04-03 11:17:37,868 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Paimon'} at: file:/opt/aspire/bigdata/seatunnel-2.3.9/connectors/connector-paimon-2.3.9.jar 2025-04-03 11:17:37,873 WARN [o.a.s.a.c.u.ConfigUtil ] [main] - Option 'plugin_input' is a List, and it is recommended to configure it as ["string1","string2"]; we will only use ',' to split the String into a list. 2025-04-03 11:17:37,873 WARN [o.a.s.a.c.u.ConfigUtil ] [main] - Option 'plugin_input' is a List, and it is recommended to configure it as ["string1","string2"]; we will only use ',' to split the String into a list. 2025-04-03 11:17:37,874 INFO [p.MultipleTableJobConfigParser] [main] - start generating all sources. 2025-04-03 11:17:38,240 INFO [o.a.s.c.s.e.c.EsRestClient ] [main] - GET /alert_202405*/_mappings respnse={"alert_202405":{"mappings":{"his":{"properties":{"alert_end_time":{"type":"date"},"alert_explain":{"type":"keyword"},"alert_id":{"type":"keyword"},"alert_level":{"type":"keyword"},"alert_reason":{"type":"keyword"},"alert_start_month":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"alert_start_time":{"type":"date"},"alert_type":{"type":"keyword"},"biz_sys":{"type":"keyword"},"cancel_people":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"cancel_remark":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"cancel_time":{"type":"date"},"change_plan":{"type":"keyword"},"city_name":{"type":"keyword"},"classify_type":{"type":"short"},"clear_value":{"type":"keyword"},"confirm_people":{"type":"keyword"},"confirm_remark":{"type":"keyword"},"confirm_state":{"type":"keyword"},"confirm_time":{"type":"date"}, "create_time":{"type":"date"},"cur_moni_time":{"type":"date"},"cur_moni_value":{"type":"keyword"},"device_id":{"type":"keyword"},"device_name":{"type":"keyword"},"device_type":{"type":"keyword"},"device_type_name":{"type":"keyword"},"engineering_status":{"type":"keyword"},"fixed":{"type":"short"},"force_clear":{"type":"short"},"force_clear_reason":{"type":"keyword"},"full_name":{"type":"keyword"},"index_seq":{"type":"long"},"is_standard_level":{"type":"keyword"},"logical_type":{"type":"keyword"},"lsc_id":{"type":"keyword"},"lsc_name":{"type":"keyword"},"manufacturer_name":{"type":"keyword"},"mete_code":{"type":"keyword"},"mete_id":{"type":"keyword"},"mete_name":{"type":"keyword"},"namespace":{"type":"keyword"},"object_id":{"type":"keyword"},"object_type":{"type":"keyword"},"order_id":{"type":"keyword"},"order_status":{"type":"keyword"},"order_type":{"type":"keyword"},"precinct_id":{"type":"keyword"},"precinct_name":{"type":"keyword"},"province_name":{"type":"keyword"},"r_alert_id":{ "type":"keyword"},"raw_mete_code":{"type":"keyword"},"raw_mete_name":{"type":"keyword"},"remark":{"type":"keyword"},"room_id":{"type":"keyword"},"room_kind_name":{"type":"keyword"},"room_name":{"type":"keyword"},"serial_no":{"type":"keyword"},"show_type":{"type":"short"},"site_name":{"type":"keyword"},"site_type":{"type":"keyword"},"source_alert_id":{"type":"keyword"},"source_clear_alert_id":{"type":"keyword"},"source_room":{"type":"keyword"},"status_masks":{"type":"keyword"},"sub_logical_type":{"type":"keyword"},"task_flag":{"type":"keyword"},"third_party_flag":{"type":"short"},"threshold":{"type":"keyword"}}}}}} 2025-04-03 11:17:38,248 WARN [o.a.s.c.s.e.c.EsRestClient ] [main] - fail to get elasticsearch field _id mapping type,so give a default type text 2025-04-03 11:17:38,248 WARN [o.a.s.c.s.e.c.EsRestClient ] [main] - fail to get elasticsearch field rce_clear_alert_id mapping type,so give a default type text 2025-04-03 11:17:38,258 INFO [o.a.s.a.t.f.FactoryUtil ] [main] - get the CatalogTable from source Elasticsearch: elasticsearch.null.alert_202405* 2025-04-03 11:17:38,348 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSource Plugin from /opt/aspire/bigdata/seatunnel-2.3.9/connectors 2025-04-03 11:17:38,354 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Elasticsearch'} at: file:/opt/aspire/bigdata/seatunnel-2.3.9/connectors/connector-elasticsearch-2.3.9.jar 2025-04-03 11:17:38,356 INFO [p.MultipleTableJobConfigParser] [main] - start generating all transforms. 2025-04-03 11:17:38,359 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelTransform Plugin from /opt/aspire/bigdata/seatunnel-2.3.9/connectors 2025-04-03 11:17:38,359 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='transform', pluginName='Sql'} at: file:/opt/aspire/bigdata/seatunnel-2.3.9/connectors/seatunnel-transforms-v2-2.3.9.jar 2025-04-03 11:17:38,360 WARN [o.a.s.a.c.u.ConfigUtil ] [main] - Option 'plugin_input' is a List, and it is recommended to configure it as ["string1","string2"]; we will only use ',' to split the String into a list. 2025-04-03 11:17:38,376 WARN [o.a.s.a.c.u.ConfigUtil ] [main] - Option 'plugin_input' is a List, and it is recommended to configure it as ["string1","string2"]; we will only use ',' to split the String into a list. 2025-04-03 11:17:38,474 INFO [p.MultipleTableJobConfigParser] [main] - start generating all sinks. 2025-04-03 11:17:38,475 WARN [o.a.s.a.c.u.ConfigUtil ] [main] - Option 'plugin_input' is a List, and it is recommended to configure it as ["string1","string2"]; we will only use ',' to split the String into a list. 2025-04-03 11:17:38,489 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /opt/aspire/bigdata/seatunnel-2.3.9/connectors 2025-04-03 11:17:38,490 INFO [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='Paimon'} at: file:/opt/aspire/bigdata/seatunnel-2.3.9/connectors/connector-paimon-2.3.9.jar 2025-04-03 11:17:38,500 INFO [o.a.s.a.t.f.FactoryUtil ] [main] - Create sink 'Paimon' with upstream input catalog-table[database: null, schema: null, table: alert_202405*] 2025-04-03 11:17:38,665 INFO [.c.s.p.s.PaimonSecurityContext] [main] - Hadoop config initialized: org.apache.seatunnel.connectors.seatunnel.paimon.config.PaimonHadoopConfiguration 2025-04-03 11:17:38,725 INFO [o.a.s.e.c.j.ClientJobProxy ] [main] - Start submit job, job id: 960015646215634949, with plugin jar [file:/opt/aspire/bigdata/seatunnel-2.3.9/connectors/seatunnel-transforms-v2-2.3.9.jar, file:/opt/aspire/bigdata/seatunnel-2.3.9/connectors/connector-elasticsearch-2.3.9.jar, file:/opt/aspire/bigdata/seatunnel-2.3.9/connectors/connector-paimon-2.3.9.jar] 2025-04-03 11:17:38,800 INFO [o.a.s.e.c.j.ClientJobProxy ] [main] - Submit job finished, job id: 960015646215634949, job name: SeaTunnel_Job 2025-04-03 11:17:38,810 INFO [o.a.s.e.c.j.JobStatusRunner ] [job-status-runner-960015646215634949] - Job Id : 960015646215634949 enter pending queue, current status:PENDING ,please wait task schedule 2025-04-03 11:17:38,810 WARN [o.a.s.e.c.j.JobMetricsRunner ] [job-metrics-runner-960015646215634949] - Failed to get job metrics summary, it maybe first-run 2025-04-03 11:17:43,812 INFO [o.a.s.e.c.j.JobStatusRunner ] [job-status-runner-960015646215634949] - Job ID: 960015646215634949 has been scheduled and entered the next state. Current status: RUNNING 2025-04-03 11:18:47,972 INFO [o.a.s.e.c.j.JobMetricsRunner ] [job-metrics-runner-960015646215634949] - *********************************************** Job Progress Information *********************************************** Job Id : 960015646215634949 Read Count So Far : 30000 Write Count So Far : 30000 Average Read Count : 434/s Average Write Count : 434/s Last Statistic Time : 2025-04-03 11:17:38 Current Statistic Time : 2025-04-03 11:18:47 *********************************************** 2025-04-03 11:19:44,611 INFO [o.a.s.e.c.j.JobMetricsRunner ] [job-metrics-runner-960015646215634949] - *********************************************** Job Progress Information *********************************************** Job Id : 960015646215634949 Read Count So Far : 50000 Write Count So Far : 50000 Average Read Count : 357/s Average Write Count : 357/s Last Statistic Time : 2025-04-03 11:18:47 Current Statistic Time : 2025-04-03 11:19:44 *********************************************** ``` ### Error Exception ```log 同步效率低，且不稳定 *********************************************** Job Progress Information *********************************************** Job Id : 960015646215634949 Read Count So Far : 331481 Write Count So Far : 331482 Average Read Count : 2064/s Average Write Count : 2064/s Last Statistic Time : 2025-04-03 11:23:46 Current Statistic Time : 2025-04-03 11:24:40 *********************************************** 2025-04-03 11:25:39,002 INFO [o.a.s.e.c.j.JobMetricsRunner ] [job-metrics-runner-960015646215634949] - *********************************************** Job Progress Information *********************************************** Job Id : 960015646215634949 Read Count So Far : 350000 Write Count So Far : 350000 Average Read Count : 319/s Average Write Count : 319/s Last Statistic Time : 2025-04-03 11:24:40 Current Statistic Time : 2025-04-03 11:25:39 *********************************************** ``` ### Zeta or Flink or Spark Version Zeta ### Java or Scala Version jdk1.8.0_201 ### Screenshots _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
