huangyutongs opened a new issue, #9423: URL: https://github.com/apache/seatunnel/issues/9423
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened I deployed seatunnel in separate mode on k8s and exposed the ui through ingress. The following error was thrown when querying the log  Using the API request log on the master node throws an error ``` curl http://localhost:8080/logs/985019895638982659?format=json -uadmin:xxx {"status":"fail","message":"Json deserialization exception."} ``` The logs can be found in the log directory ``` bash-5.2# ls /opt/seatunnel/logs/job-985019895638982659.log /opt/seatunnel/logs/job-985019895638982659.log ``` This interface can successfully obtain logs ``` curl http://localhost:8080/logs/job-985019895638982659.log -uadmin:xxx ``` ### SeaTunnel Version 2.3.11 ### SeaTunnel Config ```conf kind: ConfigMap apiVersion: v1 metadata: name: seatunnelmap-master data: seatunnel.yaml: | seatunnel: engine: http: enable-http: true port: 8080 enable-basic-auth: true basic-auth-username: "admin" basic-auth-password: "xxxxxxxxx" slot-service: slot-allocation-strategy: SYSTEM_LOAD job-schedule-strategy: WAIT classloader-cache-mode: true history-job-expire-minutes: 1440 backup-count: 1 queue-type: blockingqueue print-execution-info-interval: 60 print-job-metrics-info-interval: 60 # slot-service: # dynamic-slot: true checkpoint: interval: 10000 timeout: 60000 storage: type: hdfs max-retained: 3 plugin-config: storage.type: oss namespace: /seatunnel_checkpoint_snapshot oss.bucket: oss://seatunnel-xxxxxx-snapshot fs.oss.accessKeyId: xxxxxxxxxxxxxxxx fs.oss.accessKeySecret: xxxxxxxxxxxxx fs.oss.endpoint: oss-cn-shanghai.aliyuncs.com telemetry: metric: enabled: false --- kind: ConfigMap apiVersion: v1 metadata: name: seatunnelmap-worker data: seatunnel.yaml: | seatunnel: engine: job-schedule-strategy: WAIT classloader-cache-mode: true history-job-expire-minutes: 1440 # backup-count: 1 queue-type: blockingqueue print-execution-info-interval: 60 print-job-metrics-info-interval: 60 slot-service: dynamic-slot: true slot-allocation-strategy: SYSTEM_LOAD # slot-num: 4 # checkpoint: # interval: 10000 # timeout: 60000 # storage: # type: hdfs # max-retained: 3 # plugin-config: # namespace: /tmp/seatunnel/checkpoint_snapshot # storage.type: hdfs # fs.defaultFS: file:///tmp/ # Ensure that the directory has written permission telemetry: metric: enabled: false --- kind: ConfigMap apiVersion: v1 metadata: name: hazelcast-master data: hazelcast-master.yaml: | hazelcast: cluster-name: seatunnel network: rest-api: enabled: true endpoint-groups: CLUSTER_WRITE: enabled: true DATA: enabled: true join: tcp-ip: enabled: true member-list: - seatunnel-master-0.seatunnel-master.kube-pre-release.svc.cluster.local:5801 - seatunnel-master-1.seatunnel-master.kube-pre-release.svc.cluster.local:5801 - seatunnel-master-2.seatunnel-master.kube-pre-release.svc.cluster.local:5801 - seatunnel-worker-0.seatunnel-worker.kube-pre-release.svc.cluster.local:5801 - seatunnel-worker-1.seatunnel-worker.kube-pre-release.svc.cluster.local:5801 - seatunnel-worker-2.seatunnel-worker.kube-pre-release.svc.cluster.local:5801 port: auto-increment: false port: 5801 properties: hazelcast.heartbeat.failuredetector.type: phi-accrual hazelcast.heartbeat.interval.seconds: 2 hazelcast.max.no.heartbeat.seconds: 180 hazelcast.heartbeat.phiaccrual.failuredetector.threshold: 10 hazelcast.heartbeat.phiaccrual.failuredetector.sample.size: 200 hazelcast.heartbeat.phiaccrual.failuredetector.min.std.dev.millis: 100 hazelcast.invocation.max.retry.count: 20 hazelcast.tcp.join.port.try.count: 30 hazelcast.logging.type: log4j2 hazelcast.operation.generic.thread.count: 50 --- kind: ConfigMap apiVersion: v1 metadata: name: hazelcast-worker data: hazelcast-worker.yaml: | hazelcast: cluster-name: seatunnel network: join: tcp-ip: enabled: true member-list: - seatunnel-master-0.seatunnel-master.kube-pre-release.svc.cluster.local:5801 - seatunnel-master-1.seatunnel-master.kube-pre-release.svc.cluster.local:5801 - seatunnel-master-2.seatunnel-master.kube-pre-release.svc.cluster.local:5801 - seatunnel-worker-0.seatunnel-worker.kube-pre-release.svc.cluster.local:5801 - seatunnel-worker-1.seatunnel-worker.kube-pre-release.svc.cluster.local:5801 - seatunnel-worker-2.seatunnel-worker.kube-pre-release.svc.cluster.local:5801 port: auto-increment: false port: 5801 properties: hazelcast.heartbeat.failuredetector.type: phi-accrual hazelcast.heartbeat.interval.seconds: 2 hazelcast.max.no.heartbeat.seconds: 180 hazelcast.heartbeat.phiaccrual.failuredetector.threshold: 10 hazelcast.heartbeat.phiaccrual.failuredetector.sample.size: 200 hazelcast.heartbeat.phiaccrual.failuredetector.min.std.dev.millis: 100 hazelcast.invocation.max.retry.count: 20 hazelcast.tcp.join.port.try.count: 30 hazelcast.logging.type: log4j2 hazelcast.operation.generic.thread.count: 50 --- cat log4j2.properties # The minimum amount of time, in seconds, that must elapse before the file configuration is checked for changes. monitorInterval = 60 property.file_path = ${sys:seatunnel.logs.path:-/tmp/seatunnel/logs} property.file_name = ${sys:seatunnel.logs.file_name:-seatunnel} property.file_split_size = 100MB property.file_count = 100 property.file_ttl = 7d rootLogger.level = INFO logger.zeta.name=org.apache.seatunnel.engine logger.zeta.level=INFO logger.debezium.name=io.debezium.connector logger.debezium.level=WARN ############################ log output to console ############################# #rootLogger.appenderRef.consoleStdout.ref = consoleStdoutAppender #rootLogger.appenderRef.consoleStderr.ref = consoleStderrAppender ############################ log output to console ############################# ############################ log output to file ############################# rootLogger.appenderRef.file.ref = routingAppender ############################ log output to file ############################# appender.consoleStdout.name = consoleStdoutAppender appender.consoleStdout.type = CONSOLE appender.consoleStdout.target = SYSTEM_OUT appender.consoleStdout.layout.type = PatternLayout appender.consoleStdout.layout.pattern = [%X{ST-JID}] %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%-30.30c{1.}] [%t] - %m%n appender.consoleStdout.filter.acceptLtWarn.type = ThresholdFilter appender.consoleStdout.filter.acceptLtWarn.level = WARN appender.consoleStdout.filter.acceptLtWarn.onMatch = DENY appender.consoleStdout.filter.acceptLtWarn.onMismatch = ACCEPT appender.consoleStderr.name = consoleStderrAppender appender.consoleStderr.type = CONSOLE appender.consoleStderr.target = SYSTEM_ERR appender.consoleStderr.layout.type = PatternLayout appender.consoleStderr.layout.pattern = [%X{ST-JID}] %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%-30.30c{1.}] [%t] - %m%n appender.consoleStderr.filter.acceptGteWarn.type = ThresholdFilter appender.consoleStderr.filter.acceptGteWarn.level = WARN appender.consoleStderr.filter.acceptGteWarn.onMatch = ACCEPT appender.consoleStderr.filter.acceptGteWarn.onMismatch = DENY appender.routing.name = routingAppender appender.routing.type = Routing appender.routing.purge.type = IdlePurgePolicy appender.routing.purge.timeToLive = 60 appender.routing.purge.checkInterval = 1 appender.routing.route.type = Routes appender.routing.route.pattern = $${ctx:ST-JID} appender.routing.route.system.type = Route appender.routing.route.system.key = $${ctx:ST-JID} appender.routing.route.system.ref = fileAppender appender.routing.route.job.type = Route appender.routing.route.job.appender.type = File appender.routing.route.job.appender.name = job-${ctx:ST-JID} appender.routing.route.job.appender.fileName = ${file_path}/job-${ctx:ST-JID}.log appender.routing.route.job.appender.layout.type = PatternLayout appender.routing.route.job.appender.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%-30.30c{1.}] [%t] - %m%n appender.file.name = fileAppender appender.file.type = RollingFile appender.file.fileName = ${file_path}/${file_name}.log appender.file.filePattern = ${file_path}/${file_name}.log.%d{yyyy-MM-dd}-%i appender.file.append = true appender.file.layout.type = PatternLayout appender.file.layout.pattern = [%X{ST-JID}] %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%-30.30c{1.}] [%t] - %m%n appender.file.policies.type = Policies appender.file.policies.time.type = TimeBasedTriggeringPolicy appender.file.policies.time.modulate = true appender.file.policies.size.type = SizeBasedTriggeringPolicy appender.file.policies.size.size = ${file_split_size} appender.file.strategy.type = DefaultRolloverStrategy appender.file.strategy.fileIndex = nomax appender.file.strategy.action.type = Delete appender.file.strategy.action.basepath = ${file_path} appender.file.strategy.action.maxDepth = 1 appender.file.strategy.action.condition.type = IfFileName appender.file.strategy.action.condition.glob = ${file_name}.log* appender.file.strategy.action.condition.nested_condition.type = IfAny appender.file.strategy.action.condition.nested_condition.lastModify.type = IfLastModified appender.file.strategy.action.condition.nested_condition.lastModify.age = ${file_ttl} appender.file.strategy.action.condition.nested_condition.fileCount.type = IfAccumulatedFileCount appender.file.strategy.action.condition.nested_condition.fileCount.exceeds = ${file_count} ``` ### Running Command ```shell /opt/seatunnel/bin/seatunnel-cluster.sh -DJvmOption="-Xms2G -Xmx2G -XX:+UseG1GC -XX:MaxGCPauseMillis=100" -r master /opt/seatunnel/bin/seatunnel-cluster.sh -DJvmOption="-Xms2G -Xmx2G -XX:+UseG1GC -XX:MaxGCPauseMillis=100" -r worker 3 master nodes, 3 worker nodes ``` ### Error Exception ```log [] 2025-06-11 03:00:09,203 ERROR [.s.r.f.ExceptionHandlingFilter] [qtp1898730595-209] - Error occurred while processing request java.lang.RuntimeException: Json deserialization exception. at org.apache.seatunnel.common.utils.JsonUtils.parseArray(JsonUtils.java:279) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.engine.server.rest.service.LogService.lambda$allLogNameList$1(LogService.java:76) ~[seatunnel-starter.jar:2.3.11] at java.lang.Iterable.forEach(Iterable.java:75) ~[?:1.8.0_452] at org.apache.seatunnel.engine.server.rest.service.LogService.allLogNameList(LogService.java:70) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.engine.server.rest.service.LogService.allNodeLogFormatJson(LogService.java:97) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.engine.server.rest.servlet.AllNodeLogServlet.doGet(AllNodeLogServlet.java:69) ~[seatunnel-starter.jar:2.3.11] at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.11] at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) ~[seatunnel-hadoop3-3.1.4-uber.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.engine.server.rest.filter.BasicAuthFilter.doFilter(BasicAuthFilter.java:90) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.engine.server.rest.filter.ExceptionHandlingFilter.doFilter(ExceptionHandlingFilter.java:50) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.Server.handle(Server.java:516) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) ~[seatunnel-starter.jar:2.3.11] at java.lang.Thread.run(Thread.java:750) [?:1.8.0_452] Caused by: java.lang.IllegalArgumentException: argument "content" is null at org.apache.seatunnel.shade.com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4829) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.shade.com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3074) ~[seatunnel-starter.jar:2.3.11] at org.apache.seatunnel.common.utils.JsonUtils.parseArray(JsonUtils.java:277) ~[seatunnel-starter.jar:2.3.11] ... 43 more ``` ### Zeta or Flink or Spark Version _No response_ ### Java or Scala Version Use the amazoncorretto:8-al2023-jre base image java -version openjdk version "1.8.0_452" OpenJDK Runtime Environment Corretto-8.452.09.2 (build 1.8.0_452-b09) OpenJDK 64-Bit Server VM Corretto-8.452.09.2 (build 25.452-b09, mixed mode) ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
