[
https://issues.apache.org/jira/browse/SPARK-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheng Lian updated SPARK-15983:
---
Description:
Interface method {{FileFormat.prepareRead()}} was added in [PR
#12088|https://github.com/apache/spark/pull/12088] to handle a special case in
the LibSVM data source.
However, the semantics of this interface method isn't intuitive: it returns a
modified version of the data source options map. Considering that the LibSVM
case can be easily handled using schema metadata inside {{inferSchema}}, we can
remove this interface method to keep the {{FileFormat}} interface clean.
was:
Interface method {{FileFormat.prepareRead()}} was added in [PR
#12088|https://github.com/apache/spark/pull/12088] to handle a special case in
the LibSVM data source.
However, the semantics of this interface method isn't intuitive: it returns a
modified version of the data source options map. Considering that the LibSVM
case can be easily handled using schema metadata inside inferSchema, we can
remove this interface method to keep the FileFormat interface clean.
> Remove FileFormat.prepareRead
> -
>
> Key: SPARK-15983
> URL: https://issues.apache.org/jira/browse/SPARK-15983
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 2.0.0
>Reporter: Cheng Lian
>Assignee: Cheng Lian
>
> Interface method {{FileFormat.prepareRead()}} was added in [PR
> #12088|https://github.com/apache/spark/pull/12088] to handle a special case
> in the LibSVM data source.
> However, the semantics of this interface method isn't intuitive: it returns a
> modified version of the data source options map. Considering that the LibSVM
> case can be easily handled using schema metadata inside {{inferSchema}}, we
> can remove this interface method to keep the {{FileFormat}} interface clean.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
