[
https://issues.apache.org/jira/browse/SPARK-25104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Yuanjian updated SPARK-25104:
--------------------------------
Description:
With code changes in
[https://github.com/apache/spark/pull/21847|https://github.com/apache/spark/pull/21847]
, Spark can write out data as per user provided output schema.
To make it more robust and user friendly, we should validate the Avro schema
before tasks launched.
Also we should support output logical decimal type as BYTES (By default we
output as FIXED)
was:
With code changes in
[https://github.com/apache/spark/pull/21847|https://github.com/apache/spark/pull/21847,]
, Spark can write out data as per user provided output schema.
To make it more robust and user friendly, we should validate the Avro schema
before tasks launched.
Also we should support output logical decimal type as BYTES (By default we
output as FIXED)
> Validate user specified output schema
> -------------------------------------
>
> Key: SPARK-25104
> URL: https://issues.apache.org/jira/browse/SPARK-25104
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Gengliang Wang
> Assignee: Gengliang Wang
> Priority: Major
> Fix For: 2.4.0
>
>
> With code changes in
> [https://github.com/apache/spark/pull/21847|https://github.com/apache/spark/pull/21847]
> , Spark can write out data as per user provided output schema.
> To make it more robust and user friendly, we should validate the Avro schema
> before tasks launched.
> Also we should support output logical decimal type as BYTES (By default we
> output as FIXED)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
