MaxGekk commented on a change in pull request #24437: [SPARK-27512][SQL] Avoid
to replace ',' in CSV's decimal type inference for backward compatibility
URL: https://github.com/apache/spark/pull/24437#discussion_r277334173
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVInferSchema.scala
##########
@@ -32,7 +34,10 @@ class CSVInferSchema(val options: CSVOptions) extends
Serializable {
options.zoneId,
options.locale)
- private val decimalParser = {
+ private val decimalParser = if (options.locale == Locale.US) {
Review comment:
It looks slightly ugly that we handle special case of `Locale.US` for JSON
inside of `ExprUtils.getDecimalParser` but for CSV outside of it. Maybe we
unify the implementation, and handle the special cases outside of generic
implementation `ExprUtils.getDecimalParser` (or process both special cases
inside of it)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
