voyagertanyao opened a new issue, #5074: URL: https://github.com/apache/seatunnel/issues/5074
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened using seatunnel 2.3.2 default engine zeta, collect hive table to hdfs, (CDH-6.2.1), run failed using connector-hive-2.3.2, connector-file-hadoop-2.3.2 **_finally, I found why the error came out. Because both these two connnector jar have the class ParquetWriteStrategy.class connector-hive-2.3.2/org/apache/seatunnel/connectors/seatunnel/file/sink/writer/xxxxx.class_** ### SeaTunnel Version 2.3.2 ### SeaTunnel Config ```conf env { execution.parallelism = 1 job.mode = "BATCH" } source { Hive { table_name = "pro_smb_snapshot.c_device" metastore_uri = "thrift://slave002:9083" read_partitions = ["dt=2023-07-12"] } } sink { HdfsFile { fs.defaultFS = "hdfs://hadoop001:8020" path = "/tmp/seatunnel/prof_smb/c_device" have_partition = no batch_size = 10000000 custom_filename = true file_name_expression = "${transactionId}_${now}" filename_time_format = "yyyy-MM-dd" file_format_type = "parquet" } } ``` ### Running Command ```shell bin/seatunnel.sh --config hive.yaml -e local bin/seatunnel.sh --config hive.yaml -e cluster ``` ### Error Exception ```log 2023-07-13 14:22:27,104 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-2.3.2] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED 2023-07-13 14:22:27,106 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-2.3.2] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN 2023-07-13 14:22:27,106 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed SeaTunnel client...... 2023-07-13 14:22:27,106 ERROR org.apache.seatunnel.core.starter.SeaTunnel - =============================================================================== 2023-07-13 14:22:27,106 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Fatal Error, 2023-07-13 14:22:27,106 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Please submit bug report in https://github.com/apache/seatunnel/issues 2023-07-13 14:22:27,107 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Reason:SeaTunnel job executed failed 2023-07-13 14:22:27,109 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:188) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: java.util.concurrent.CompletionException: com.hazelcast.nio.serialization.HazelcastSerializationException: java.io.InvalidClassException: org.apache.seatunnel.connectors.seatunnel.file.sink.writer.ParquetWriteStrategy; local class incompatible: stream classdesc serialVersionUID = 3902709188302080196, local class serialVersionUID = 3615746833076381224 at com.hazelcast.spi.impl.AbstractInvocationFuture.wrapInCompletionException(AbstractInvocationFuture.java:1347) at com.hazelcast.spi.impl.AbstractInvocationFuture.cascadeException(AbstractInvocationFuture.java:1340) at com.hazelcast.spi.impl.AbstractInvocationFuture.access$200(AbstractInvocationFuture.java:65) at com.hazelcast.spi.impl.AbstractInvocationFuture$ApplyNode.execute(AbstractInvocationFuture.java:1478) at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockOtherNode(AbstractInvocationFuture.java:797) at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockAll(AbstractInvocationFuture.java:759) at com.hazelcast.spi.impl.AbstractInvocationFuture.complete0(AbstractInvocationFuture.java:1235) at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionallyInternal(AbstractInvocationFuture.java:1223) at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionally(AbstractInvocationFuture.java:709) at com.hazelcast.client.impl.spi.impl.ClientInvocation.completeExceptionally(ClientInvocation.java:294) at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyExceptionWithOwnedPermission(ClientInvocation.java:321) at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyException(ClientInvocation.java:304) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.handleResponse(ClientResponseHandlerSupplier.java:164) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.process(ClientResponseHandlerSupplier.java:141) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.access$300(ClientResponseHandlerSupplier.java:60) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:251) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:243) at com.hazelcast.client.impl.connection.tcp.TcpClientConnection.handleClientMessage(TcpClientConnection.java:245) at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.handleMessage(ClientMessageDecoder.java:135) at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.onRead(ClientMessageDecoder.java:89) at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:136) at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:383) at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:368) at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:294) at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249) at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) Caused by: com.hazelcast.nio.serialization.HazelcastSerializationException: java.io.InvalidClassException: org.apache.seatunnel.connectors.seatunnel.file.sink.writer.ParquetWriteStrategy; local class incompatible: stream classdesc serialVersionUID = 3902709188302080196, local class serialVersionUID = 3615746833076381224 at com.hazelcast.internal.serialization.impl.SerializationUtil.handleException(SerializationUtil.java:111) at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:355) at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:600) at org.apache.seatunnel.engine.core.dag.logical.LogicalVertex.readData(LogicalVertex.java:99) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51) at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:349) at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:600) at org.apache.seatunnel.engine.core.dag.logical.LogicalDag.readData(LogicalDag.java:154) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51) at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:268) at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject.deserializeWithCustomClassLoader(CustomClassLoadedObject.java:66) at org.apache.seatunnel.engine.server.master.JobMaster.init(JobMaster.java:203) at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$5(CoordinatorService.java:438) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.io.InvalidClassException: org.apache.seatunnel.connectors.seatunnel.file.sink.writer.ParquetWriteStrategy; local class incompatible: stream classdesc serialVersionUID = 3902709188302080196, local class serialVersionUID = 3615746833076381224 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:699) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2004) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1851) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2185) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2430) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2354) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2212) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2430) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2354) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2212) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:502) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:460) at com.hazelcast.internal.serialization.impl.defaultserializers.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:92) at com.hazelcast.internal.serialization.impl.defaultserializers.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:85) at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:349) ... 22 more 2023-07-13 14:22:27,109 ERROR org.apache.seatunnel.core.starter.SeaTunnel - =============================================================================== Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:188) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: java.util.concurrent.CompletionException: com.hazelcast.nio.serialization.HazelcastSerializationException: java.io.InvalidClassException: org.apache.seatunnel.connectors.seatunnel.file.sink.writer.ParquetWriteStrategy; local class incompatible: stream classdesc serialVersionUID = 3902709188302080196, local class serialVersionUID = 3615746833076381224 at com.hazelcast.spi.impl.AbstractInvocationFuture.wrapInCompletionException(AbstractInvocationFuture.java:1347) at com.hazelcast.spi.impl.AbstractInvocationFuture.cascadeException(AbstractInvocationFuture.java:1340) at com.hazelcast.spi.impl.AbstractInvocationFuture.access$200(AbstractInvocationFuture.java:65) at com.hazelcast.spi.impl.AbstractInvocationFuture$ApplyNode.execute(AbstractInvocationFuture.java:1478) at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockOtherNode(AbstractInvocationFuture.java:797) at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockAll(AbstractInvocationFuture.java:759) at com.hazelcast.spi.impl.AbstractInvocationFuture.complete0(AbstractInvocationFuture.java:1235) at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionallyInternal(AbstractInvocationFuture.java:1223) at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionally(AbstractInvocationFuture.java:709) at com.hazelcast.client.impl.spi.impl.ClientInvocation.completeExceptionally(ClientInvocation.java:294) at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyExceptionWithOwnedPermission(ClientInvocation.java:321) at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyException(ClientInvocation.java:304) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.handleResponse(ClientResponseHandlerSupplier.java:164) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.process(ClientResponseHandlerSupplier.java:141) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.access$300(ClientResponseHandlerSupplier.java:60) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:251) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:243) at com.hazelcast.client.impl.connection.tcp.TcpClientConnection.handleClientMessage(TcpClientConnection.java:245) at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.handleMessage(ClientMessageDecoder.java:135) at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.onRead(ClientMessageDecoder.java:89) at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:136) at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:383) at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:368) at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:294) at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249) at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) Caused by: com.hazelcast.nio.serialization.HazelcastSerializationException: java.io.InvalidClassException: org.apache.seatunnel.connectors.seatunnel.file.sink.writer.ParquetWriteStrategy; local class incompatible: stream classdesc serialVersionUID = 3902709188302080196, local class serialVersionUID = 3615746833076381224 at com.hazelcast.internal.serialization.impl.SerializationUtil.handleException(SerializationUtil.java:111) at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:355) at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:600) at org.apache.seatunnel.engine.core.dag.logical.LogicalVertex.readData(LogicalVertex.java:99) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51) at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:349) at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:600) at org.apache.seatunnel.engine.core.dag.logical.LogicalDag.readData(LogicalDag.java:154) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106) at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51) at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:268) at com.hazelcast.jet.impl.execution.init.CustomClassLoadedObject.deserializeWithCustomClassLoader(CustomClassLoadedObject.java:66) at org.apache.seatunnel.engine.server.master.JobMaster.init(JobMaster.java:203) at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$5(CoordinatorService.java:438) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.io.InvalidClassException: org.apache.seatunnel.connectors.seatunnel.file.sink.writer.ParquetWriteStrategy; local class incompatible: stream classdesc serialVersionUID = 3902709188302080196, local class serialVersionUID = 3615746833076381224 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:699) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2004) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1851) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2185) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2430) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2354) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2212) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2430) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2354) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2212) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1668) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:502) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:460) at com.hazelcast.internal.serialization.impl.defaultserializers.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:92) at com.hazelcast.internal.serialization.impl.defaultserializers.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:85) at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:44) at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:349) ... 22 more ``` ### Flink or Spark Version _No response_ ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
