jochenshen24 opened a new issue, #4957: URL: https://github.com/apache/seatunnel/issues/4957
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened 同步达梦数据库到mysql, 报错：Caused by: org.apache.seatunnel.api.common.PrepareFailException: ErrorCode:[API-01], ErrorDescription:[Configuration item validate failed] - PluginName: jdbc, PluginType: source, Message: org.apache.seatunnel.connectors.seatunnel.jdbc.exception.JdbcConnectorException: ErrorCode:[COMMON-06], ErrorDescription:[Illegal argument] - field "id" not contain in query select * from t_1。 达梦数据库开启了大小写敏感参数 select case_sensitive <img width="90" alt="image" src="https://github.com/apache/seatunnel/assets/105631880/9d4b1215-1217-44d9-882f-45f73309ea52";> 数据表SQL： CREATE TABLE "SHM"."t_1" ( "id" INT NOT NULL, "a" VARCHAR2(50) NOT NULL, "b" DATETIME(6) NOT NULL, NOT CLUSTER PRIMARY KEY("id")) STORAGE(ON "SHM", CLUSTERBTR) ; seatunnel配置文件，配置partition_column参数后，执行异常；数据库关闭case_sensitive 大小写敏感后正常。 ### SeaTunnel Version 2.3.1 ### SeaTunnel Config ```conf env { # You can set SeaTunnel environment configuration here execution.parallelism = 2 job.mode = "BATCH" #checkpoint.interval = 10000 #execution.checkpoint.interval = 10000 #execution.checkpoint.data-uri = "hdfs://localhost:9000/checkpoint" } source { # This is a example source plugin **only for test and demonstrate the feature source plugin** Jdbc { driver = "dm.jdbc.driver.DmDriver" url = "jdbc:dm://192.168.0.13:5236?schema=shm" user = "xxx" password = "xxx" partition_column = id query = "select * from t_1" } # If you would like to get more information about how to configure Seatunnel and see full list of source plugins, # please go to https://seatunnel.apache.org/docs/category/source-v2 } sink { Jdbc { driver = "com.mysql.cj.jdbc.Driver" url = "jdbc:mysql://xxx:xxx/test?useUnicode=true&characterEncoding=utf8&useSSL=false" user = "xxx" password = "xxx" database = "test" table = "test" primary_keys = id batch_size = 1000 } # If you would like to get more information about how to configure Seatunnel and see full list of sink plugins, # please go to https://seatunnel.apache.org/docs/category/sink-v2 } ``` ### Running Command ```shell bin/seatunnel.sh --config config/1.config -m local ``` ### Error Exception ```log 2023-06-20 19:41:46,892 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load SeaTunnelSource Plugin from /data/applications/apache-seatunnel-incubating-2.3.1/connectors/seatunnel 2023-06-20 19:41:46,902 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Discovery plugin jar: Jdbc at: file:/data/applications/apache-seatunnel-incubating-2.3.1/connectors/seatunnel/connector-jdbc-2.3.1.jar 2023-06-20 19:41:46,912 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load plugin: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Jdbc'} from path: file:/data/applications/apache-seatunnel-incubating-2.3.1/connectors/seatunnel/connector-jdbc-2.3.1.jar use classloader: org.apache.seatunnel.engine.common.loader.SeaTunnelChildFirstClassLoader 2023-06-20 19:41:47,350 WARN org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSource - get row type info exception dm.jdbc.driver.DMException: 第1 行附近出现错误: 无效的表或视图名[T_1] at dm.jdbc.driver.DBError.throwException(DBError.java:711) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.a.b.o.x(MSG.java:591) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.a.b.o.C(MSG.java:526) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.a.b.o.B(MSG.java:507) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.a.a.a(DBAccess.java:245) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.a.a.a(DBAccess.java:718) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.driver.DmdbPreparedStatement.prepareSql(DmdbPreparedStatement.java:303) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.driver.DmdbPreparedStatement.allocateHandle(DmdbPreparedStatement.java:229) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.driver.DmdbPreparedStatement.<init>(DmdbPreparedStatement.java:131) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.driver.DmdbPreparedStatement.<init>(DmdbPreparedStatement.java:181) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.driver.DmdbConnection.do_prepareStatement(DmdbConnection.java:722) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.driver.DmdbConnection.do_prepareStatement(DmdbConnection.java:708) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at dm.jdbc.driver.DmdbConnection.prepareStatement(DmdbConnection.java:1332) ~[DmJdbcDriver18-8.1.2.192.jar:- 8.1.2.192 - Production] at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.dialect.JdbcDialect.getResultSetMetaData(JdbcDialect.java:193) ~[connector-jdbc-2.3.1.jar:2.3.1] at org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSource.initTableField(JdbcSource.java:152) [connector-jdbc-2.3.1.jar:2.3.1] at org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSource.prepare(JdbcSource.java:91) [connector-jdbc-2.3.1.jar:2.3.1] at org.apache.seatunnel.engine.core.parse.ConnectorInstanceLoader.loadSourceInstance(ConnectorInstanceLoader.java:64) [seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.core.parse.JobConfigParser.sampleAnalyze(JobConfigParser.java:371) [seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.core.parse.JobConfigParser.parse(JobConfigParser.java:132) [seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:112) [seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getLogicalDag(JobExecutionEnvironment.java:155) [seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:147) [seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:140) [seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) [seatunnel-starter.jar:2.3.1] at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) [seatunnel-starter.jar:2.3.1] 2023-06-20 19:41:47,357 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-260464] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN 2023-06-20 19:41:47,362 INFO com.hazelcast.internal.server.tcp.TcpServerConnection - [localhost]:5801 [seatunnel-260464] [5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:43114, qualifier=null, endpoint=[127.0.0.1]:43114, remoteUuid=dafc4752-5c7c-46c5-aec7-400e6279e134, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side 2023-06-20 19:41:47,364 INFO com.hazelcast.client.impl.ClientEndpointManager - [localhost]:5801 [seatunnel-260464] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /127.0.0.1:5801->/127.0.0.1:43114, qualifier=null, endpoint=[127.0.0.1]:43114, remoteUuid=dafc4752-5c7c-46c5-aec7-400e6279e134, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=dafc4752-5c7c-46c5-aec7-400e6279e134, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1687261306666, latest clientAttributes=lastStatisticsCollectionTime=1687261306709,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1687261306651,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=4900560896,os.freePhysicalMemorySize=298549248,os.freeSwapSpaceSize=8455712768,os.maxFileDescriptorCount=4096,os.openFileDescriptorCount=53,os.processCpuTime=11510000000,os.systemLoadAverage=0.12,os.totalPhysicalMemorySize=8201256960,os.tot alSwapSpaceSize=8455712768,runtime.availableProcessors=5,runtime.freeMemory=180677216,runtime.maxMemory=477626368,runtime.totalMemory=218103808,runtime.uptime=4546,runtime.usedMemory=37426592, labels=[]} 2023-06-20 19:41:47,365 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-260464] [5.1] Removed connection to endpoint: [localhost]:5801:ab23d2d5-3eaf-4a83-a6b9-5aa336a10a51, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:43114->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2023-06-20 19:41:46.746, lastWriteTime=2023-06-20 19:41:46.744, closedTime=2023-06-20 19:41:47.360, connected server version=5.1} 2023-06-20 19:41:47,365 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-260464] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED 2023-06-20 19:41:47,370 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-260464] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN 2023-06-20 19:41:47,370 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed SeaTunnel client...... 2023-06-20 19:41:47,371 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-260464] [5.1] [localhost]:5801 is SHUTTING_DOWN 2023-06-20 19:41:47,375 INFO com.hazelcast.internal.partition.impl.MigrationManager - [localhost]:5801 [seatunnel-260464] [5.1] Shutdown request of Member [localhost]:5801 - ab23d2d5-3eaf-4a83-a6b9-5aa336a10a51 this is handled 2023-06-20 19:41:47,381 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-260464] [5.1] Shutting down connection manager... 2023-06-20 19:41:47,384 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-260464] [5.1] Shutting down node engine... 2023-06-20 19:41:47,396 INFO org.apache.seatunnel.engine.server.SeaTunnelServer - master node check interrupted 2023-06-20 19:41:50,436 INFO com.hazelcast.instance.impl.NodeExtension - [localhost]:5801 [seatunnel-260464] [5.1] Destroying node NodeExtension. 2023-06-20 19:41:50,437 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-260464] [5.1] Hazelcast Shutdown is completed in 3063 ms. 2023-06-20 19:41:50,438 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-260464] [5.1] [localhost]:5801 is SHUTDOWN 2023-06-20 19:41:50,439 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed HazelcastInstance ...... 2023-06-20 19:41:50,439 ERROR org.apache.seatunnel.core.starter.SeaTunnel - =============================================================================== 2023-06-20 19:41:50,439 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Fatal Error, 2023-06-20 19:41:50,440 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Please submit bug report in https://github.com/apache/incubator-seatunnel/issues 2023-06-20 19:41:50,440 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Reason:SeaTunnel job executed failed 2023-06-20 19:41:50,444 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: org.apache.seatunnel.api.common.PrepareFailException: ErrorCode:[API-01], ErrorDescription:[Configuration item validate failed] - PluginName: jdbc, PluginType: source, Message: org.apache.seatunnel.connectors.seatunnel.jdbc.exception.JdbcConnectorException: ErrorCode:[COMMON-06], ErrorDescription:[Illegal argument] - field id not contain in query select * from t_1 at org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSource.prepare(JdbcSource.java:96) at org.apache.seatunnel.engine.core.parse.ConnectorInstanceLoader.loadSourceInstance(ConnectorInstanceLoader.java:64) at org.apache.seatunnel.engine.core.parse.JobConfigParser.sampleAnalyze(JobConfigParser.java:371) at org.apache.seatunnel.engine.core.parse.JobConfigParser.parse(JobConfigParser.java:132) at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:112) at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getLogicalDag(JobExecutionEnvironment.java:155) at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:147) at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:140) ... 2 more 2023-06-20 19:41:50,444 ERROR org.apache.seatunnel.core.starter.SeaTunnel - =============================================================================== Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: org.apache.seatunnel.api.common.PrepareFailException: ErrorCode:[API-01], ErrorDescription:[Configuration item validate failed] - PluginName: jdbc, PluginType: source, Message: org.apache.seatunnel.connectors.seatunnel.jdbc.exception.JdbcConnectorException: ErrorCode:[COMMON-06], ErrorDescription:[Illegal argument] - field id not contain in query select * from t_1 at org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSource.prepare(JdbcSource.java:96) at org.apache.seatunnel.engine.core.parse.ConnectorInstanceLoader.loadSourceInstance(ConnectorInstanceLoader.java:64) at org.apache.seatunnel.engine.core.parse.JobConfigParser.sampleAnalyze(JobConfigParser.java:371) at org.apache.seatunnel.engine.core.parse.JobConfigParser.parse(JobConfigParser.java:132) at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:112) at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getLogicalDag(JobExecutionEnvironment.java:155) at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:147) at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:140) ... 2 more ``` ### Flink or Spark Version _No response_ ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
