Nick Dimiduk created SPARK-27599:
------------------------------------
Summary: DataFrameWriter$partitionBy should be optional when
writing to a hive table
Key: SPARK-27599
URL: https://issues.apache.org/jira/browse/SPARK-27599
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.4.1
Reporter: Nick Dimiduk
When writing to an existing, partitioned table stored in the Hive metastore,
Spark requires the call to {{saveAsTable}} to provide a value for
{{partitionedBy}}, even though that information is provided by the metastore
itself. Indeed, that information is available to Spark, as it will error if the
specified {{partitionBy}} does not match that of the table definition in
metastore.
There may be other attributes of the save call that can be retrieved from the
metastore...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
