sadikovi commented on PR #40699:
URL: https://github.com/apache/spark/pull/40699#issuecomment-1517305453
I just tried without the change and df.show() seems to return correct
results:
```scala
spark.sql(
"""
select struct('a'), map('a', 1), array(1, 2), 'a'
union all
select struct(null), map('a', null), array(null, null), null
""").show()
```
```
+---------+-----------+------------+----+
|struct(a)| map(a, 1)| array(1, 2)| a|
+---------+-----------+------------+----+
| {a}| {a -> 1}| [1, 2]| a|
| {null}|{a -> null}|[null, null]|null|
+---------+-----------+------------+----+
```
Could you provide an example of inconsistency in df.show() that this PR
addresses?
The reason I brought the CSV example is because that is typically how users
would write structs/arrays/maps to a CSV file, they would transform that to a
string, potentially via CAST. That is why I think this is a behaviour change in
writing a CSV file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
