FuYouJ opened a new issue, #7517: URL: https://github.com/apache/seatunnel/issues/7517
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened I am developing a multi-table synchronization plugin and currently running unit tests. I have noticed that configurations that work in the Zeta engine do not pass the test cases in Spark mode. Here is my configuration: ```hocon source { Elasticsearch { hosts = ["https://elasticsearch:9200";] username = "elastic" password = "elasticsearch" tls_verify_certificate = false tls_verify_hostname = false index_list = [ { index = "read_index1" query = {"range": {"c_int": {"gte": 10, "lte": 20}}} source = [ c_map, c_array, c_string, c_boolean, c_tinyint, c_smallint, c_bigint, c_float, c_double, c_decimal, c_bytes, c_int, c_date, c_timestamp] array_column = { c_array = "array<tinyint>" } } { index = "read_index1" query = {"match_all": {}} source = [ c_map, c_array, c_string, c_boolean, c_tinyint, c_smallint, c_bigint, c_float, c_double, c_decimal, c_bytes, c_int, c_date, c_timestamp] array_column = { c_array = "array<tinyint>" } } ] } } transform { } sink { Elasticsearch { hosts = ["https://elasticsearch:9200";] username = "elastic" password = "elasticsearch" tls_verify_certificate = false tls_verify_hostname = false index = "multi_source_write_test_index" index_type = "st" "schema_save_mode"="CREATE_SCHEMA_WHEN_NOT_EXIST" "data_save_mode"="APPEND_DATA" } } ``` There seems to be an issue with my configuration. I shouldn't have configured two identical indices, but this configuration runs without issues in the Zeta engine, while it throws an error in Spark mode. <img width="932" alt="image" src="https://github.com/user-attachments/assets/7f8fbe7d-5a07-4e9f-ae10-75eb22a5873c";> I believe that if configuring two identical tables has a valid use case in user-defined partitioning scenarios, then Spark needs to be changed. Alternatively, if it is not valid, then the Zeta engine needs to be modified. ### SeaTunnel Version 2.3.8 dev ### SeaTunnel Config ```conf source { Elasticsearch { hosts = ["https://elasticsearch:9200";] username = "elastic" password = "elasticsearch" tls_verify_certificate = false tls_verify_hostname = false index_list = [ { index = "read_index1" query = {"range": {"c_int": {"gte": 10, "lte": 20}}} source = [ c_map, c_array, c_string, c_boolean, c_tinyint, c_smallint, c_bigint, c_float, c_double, c_decimal, c_bytes, c_int, c_date, c_timestamp] array_column = { c_array = "array<tinyint>" } } { index = "read_index2" query = {"match_all": {}} source = [ c_map, c_array, c_string, c_boolean, c_tinyint, c_smallint, c_bigint, c_float, c_double, c_decimal, c_bytes, c_int, c_date, c_timestamp] array_column = { c_array = "array<tinyint>" } } ] } } transform { } sink { Elasticsearch { hosts = ["https://elasticsearch:9200";] username = "elastic" password = "elasticsearch" tls_verify_certificate = false tls_verify_hostname = false index = "multi_source_write_test_index" index_type = "st" "schema_save_mode"="CREATE_SCHEMA_WHEN_NOT_EXIST" "data_save_mode"="APPEND_DATA" } } ``` ### Running Command ```shell default ``` ### Error Exception ```log . ``` ### Zeta or Flink or Spark Version _No response_ ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
