Alberto created SPARK-21640:
-------------------------------
Summary: Method mode with String parameters within DataFrameWriter
is error prone
Key: SPARK-21640
URL: https://issues.apache.org/jira/browse/SPARK-21640
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.2.0
Reporter: Alberto
The following method:
{code:java}
def mode(saveMode: String): DataFrameWriter[T]
{code}
sets the SaveMode of the DataFrameWriter depending on the string that is
pass-in as parameter.
There is a java Enum with all the save modes which are Append, Overwrite,
ErrorIfExists and Ignore. In my current project I was writing some code that
was using this enum to get the string value that I use to call the mode method:
{code:java}
private[utils] val configModeAppend = SaveMode.Append.toString.toLowerCase
private[utils] val configModeErrorIfExists =
SaveMode.ErrorIfExists.toString.toLowerCase
private[utils] val configModeIgnore = SaveMode.Ignore.toString.toLowerCase
private[utils] val configModeOverwrite =
SaveMode.Overwrite.toString.toLowerCase
{code}
The configModeErrorIfExists val contains the value "errorifexists" and when I
call the saveMode method using this string it does not match. I suggest to
include "errorifexists" as a right match for the ErrorIfExists SaveMode.
Will create a PR to address this issue ASAP.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
