Github user fangshil commented on the issue:
https://github.com/apache/spark/pull/21310
To summarize our discussion in this pr:
Spark-avro is now merged into Spark as a built-in data source. Upstream
community is not merging the AvroEncoder to support Avro types in Dataset,
instead, the plan is to exposing the user-defined type API to support defining
arbitrary user types in Dataset.
The purpose of this patch is to enable ExpressionEncoder to work together
with other types of Encoders, while it seems like upstream prefers to go with
UDT. Given this, we can close this PR and we will start the discussion on UDT
in another channel
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
