kaori-seasons commented on code in PR #345:
URL:
https://github.com/apache/doris-spark-connector/pull/345#discussion_r2533080458
##########
spark-doris-connector/spark-doris-connector-base/src/main/scala/org/apache/doris/spark/util/SchemaConvertors.scala:
##########
@@ -52,7 +91,11 @@ object SchemaConvertors {
case "DECIMAL128" => DecimalType(precision, scale)
case "TIME" => DataTypes.DoubleType
case "STRING" => DataTypes.StringType
- case "ARRAY" => DataTypes.StringType
+ case "ARRAY" => ArrayType(DataTypes.StringType, containsNull = true)
Review Comment:
This will change the previous state, but the data content will remain
compatible.
#### Core Changes
Previous:
Schema: tags: string
```Value: "[\"Alice\",\"Bob\"]" (string)```
Array operations were not supported.
Now:
Schema: tags: array<string>
```Value: WrappedArray(Alice, Bob) (array)```
Explode, array_contains, etc., are supported.
#### Impact
Schema type change: StringType → ArrayType(StringType)
Value type change: String → ArrayData
Data content compatibility: Elements are still strings ["Alice", "Bob"]
I understand your concerns. User code that relies on StringType checks or
uses row.getString() will need to be adapted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
