[
https://issues.apache.org/jira/browse/SPARK-25807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-25807:
---------------------------------
Affects Version/s: (was: 2.5.0)
> Mitigate 1-based substr() confusion
> -----------------------------------
>
> Key: SPARK-25807
> URL: https://issues.apache.org/jira/browse/SPARK-25807
> Project: Spark
> Issue Type: Improvement
> Components: Java API, PySpark
> Affects Versions: 1.3.0, 2.3.2, 2.4.0, 3.0.0
> Reporter: Oron Navon
> Priority: Minor
>
> The method {{Column.substr()}} is 1-based, conforming with SQL and Hive's
> {{SUBSTRING}}, and contradicting both Python's {{substr}} and Java's
> {{substr}}, which are zero-based. Both PySpark users and Java API users
> often naturally expect a 0-based {{substr()}}. Adding to the confusion,
> {{substr()}} currently allows a {{startPos}} value of 0, which returns the
> same result as {{startPos==1}}.
> Since changing {{substr()}} to 0-based is probably NOT a reasonable option
> here, I suggest making one or more of the following changes:
> # Adding a method {{substr0}}, which would be zero-based
> # Renaming {{substr}} to {{substr1}}
> # Making the existing {{substr()}} throw an exception on {{startPos==0}},
> which should catch and alert most users who expect zero-based behavior.
> This is my first discussion on this project, apologies for any faux pas.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
