[
https://issues.apache.org/jira/browse/SPARK-17012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aseem Bansal updated SPARK-17012:
---------------------------------
Description:
Currently the option that we have in DataFrameReader is nullValue which allows
us one default. But say in our data frame we have string and integers and we
want to specify the default for strings and integers differently that is
currently not possible.
If it is done for different data types then it should be possible to allow to
specify the schema to be nullable false when inferring schema (as a new option).
was:Currently the option that we have in DataFrameReader is nullValue which
allows us one default. But say in our data frame we have string and integers
and we want to specify the default for strings and integers differently that is
currently not possible.
> Reading data frames via CSV - Allow to specify default value for integers
> -------------------------------------------------------------------------
>
> Key: SPARK-17012
> URL: https://issues.apache.org/jira/browse/SPARK-17012
> Project: Spark
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Aseem Bansal
>
> Currently the option that we have in DataFrameReader is nullValue which
> allows us one default. But say in our data frame we have string and integers
> and we want to specify the default for strings and integers differently that
> is currently not possible.
> If it is done for different data types then it should be possible to allow to
> specify the schema to be nullable false when inferring schema (as a new
> option).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
