SEPURI-SAI-KRISHNA opened a new issue, #13071: URL: https://github.com/apache/seatunnel/issues/13071
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened The `schema` option is parsed with case conversions that use the JVM default locale. Under `tr-TR` and `az-AZ`, `"i".toUpperCase()` is `"İ"` (U+0130), so any lowercase value containing an `i` no longer matches. **Field types.** `SeaTunnelDataTypeConvertorUtil.deserializeSeaTunnelDataType` upper cases the declared type before matching it against `SqlType` (`:48`, `:115`, `:128`, `:194`). Measured with the class from `dev` on 26 declarations: all parse under `ROOT`, 13 fail under `tr-TR` and `az-AZ`: `string`, `int`, `bigint`, `tinyint`, `smallint`, `time`, `timestamp`, `timestamp_tz`, `decimal(10, 2)`, `array<int>`, `map<string, int>`, `map<decimal(10,2), int>`, `binary_vector`. **Vector index keys.** `VectorIndex.IndexType.of` and `MetricType.of` (`:92`, `:108`) do the same. The schema documentation says `indexType` and `metricType` are case insensitive (`cosine`, `COSINE` and `Cosine` are all valid), but under `tr-TR` and `az-AZ` 15 lowercase values are rejected, including `ivf_flat`, `diskann`, `inverted`, `ip` and `cosine`. Upper case values (`STRING`, `COSINE`) still parse, because `I` has no special mapping. `ReadonlyConfigParser` handles the `schema` option of every connector that has one. The type parser is also used by the Assert sink rules, the Elasticsearch source, Hive, and the JsonPath, LLM and Python transforms. So such jobs fail at startup on a worker whose default locale is Turkish or Azerbaijani. Running the `seatunnel-api` unit tests in a JVM started with `-Duser.language=tr -Duser.country=TR` shows it as well: 13 existing schema parsing tests fail, for example `CatalogTableUtilTest.testSimpleSchemaParse`, `TableSchemaDiscovererTest.testDiscoverTableSchemasWithSingleSchemaFields` and `SeaTunnelDataTypeConvertorUtilTest.testNestedMapType`. ### SeaTunnel Version dev (f3e1c2ee3b) ### SeaTunnel Config ```conf schema = { fields { name = string age = int vec = FLOAT_VECTOR } constraintKeys = [ { constraintName = "vec_index" constraintType = VECTOR_INDEX_KEY constraintColumns = [ { columnName = "vec" indexType = "ivf_flat" metricType = "cosine" } ] } ] } ``` Building such a schema with `CatalogTableUtil.buildWithConfig`, the call connectors use for the `schema` option, gives `ROW<name STRING,age INT,vec FLOAT_VECTOR>` under `ROOT`. Under `tr-TR` it fails on `string`, and with the field types written in upper case it fails on `ivf_flat`. ### Running Command ```shell Any job with the schema above, on a worker started with -Duser.language=tr -Duser.country=TR ``` ### Error Exception ```log ErrorCode:[COMMON-07], ErrorDescription:['SeaTunnel' unsupported data type 'string' of 'name'] schema.constraintKeys[vec_index].constraintColumns[vec] config invalid: Invalid indexType 'ivf_flat' for column 'vec'. Valid values are: [FLAT, IVF_FLAT, IVF_SQ8, IVF_PQ, HNSW, DISKANN, AUTOINDEX, SCANN, GPU_IVF_FLAT, GPU_IVF_PQ, GPU_BRUTE_FORCE, GPU_CAGRA, BIN_FLAT, BIN_IVF_FLAT, TRIE, STL_SORT, INVERTED, SPARSE_INVERTED_INDEX, SPARSE_WAND] ``` ### Zeta or Flink or Spark Version All engines. The parsing is in `seatunnel-api`. ### Java or Scala Version Reproduced on JDK 17. ### Screenshots _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
