[
https://issues.apache.org/jira/browse/SPARK-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16012377#comment-16012377
]
Alexander Enns commented on SPARK-18359:
----------------------------------------
This is exactly why there is a possibility to set the locale when a JVM
instance is created. When we submit a Job to our cluster we have to tell Spark
that it requires DE locale as our data are coming in the corresponding format.
After the migration from 1.6 to 2.1 we had to give up usage of any type apart
of string for parsing of CSV data (sounds not like it's a correct way to me). I
can not see why exactly the changes required for SPARK-18076 have been made.
But at the moment there is no possibility to parse CSV data formatted using
locale different from US and this appears to me like a quite big restriction.
> Let user specify locale in CSV parsing
> --------------------------------------
>
> Key: SPARK-18359
> URL: https://issues.apache.org/jira/browse/SPARK-18359
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.0.0, 2.0.1
> Reporter: yannick Radji
>
> On the DataFrameReader object there no CSV-specific option to set decimal
> delimiter on comma whereas dot like it use to be in France and Europe.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
