Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18664
I think all of the involved reviewers agree this is a pretty serious design
issue. We are unable to change the behavior after we officially release it.
Thus, we have to be very very careful about the design.
Can anyone first write user facing documents to explain the proposal and
the potential impact?
Let us list a few typical scenarios
Given the users/applications containing the `Timestamp` in their `Dataset`
and their processing algorithms have the corresponding logics:
- For the **new** users/applications, they first enabled Arrow and later
hit an Arrow bug? Can they simply turn off `spark.sql.execution.arrow.enable`?
If not, what should they do?
- For the **existing** users/applications, they want to utilize Arrow for
better performance. Can they just turn on `spark.sql.execution.arrow.enable`?
What should they do?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
