Github user jmchung commented on the issue:
https://github.com/apache/spark/pull/19567
gentle ping @cloud-fan and @viirya, there are some feedbacks about the
behavior of obj to string.
``` scala
case StringType =>
(array: Object) =>
array.asInstanceOf[Array[java.lang.Object]]
.map(obj => UTF8String.fromString(obj.toString))
```
As @HyukjinKwon mentioned before, it may unsafe to covnert the object to
string directly, we'll have the `java.lang.NullPointerException` in test cases.
`StringType` is different from other types use `nullSafeConvert` method to deal
with the null, it process within `UTF8String.fromString`.
The `UTF8String.fromString` returns null and UTF8String so that we cannot
use `String.valueOf(obj)` to avoid the NPE, it will raise another errors
(because `null != "null"`):
```
- Type mapping for various types *** FAILED ***
Array("a", "null", "b") did not equal List("a", null, "b")
(PostgresIntegrationSuite.scala:120)
```
IMHO, if we don't want to make separation of match cases, we may remain
null to meet the original design, e.g.,
``` scala
case StringType =>
(array: Object) =>
array.asInstanceOf[Array[java.lang.Object]]
.map(obj => UTF8String.fromString(
if (obj == null) null else obj.toString))
```
Alternative is we keep the String and Object cases?
Would you mind give some comments? I can keep working on this :)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
