[jira] [Commented] (SPARK-28036) Built-in udf left/right has inconsistent behavior
[
https://issues.apache.org/jira/browse/SPARK-28036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16877765#comment-16877765
]
ShuMing Li commented on SPARK-28036:
[~shivuson...@gmail.com] It's not the same with `Postgres`:
* In Postgres:
** left({{str}} {{text}}, {{n}} {{int}}): Return first {{n}} characters in the
string. When {{n }}is negative, return all but last |{{n}}| characters;
** right({{str}} {{text}}, {{n}} {{int}}): Return last {{n}} characters in the
string. When {{n}} is negative, return all but first |{{n}}| characters;
* In Spark:
** left(str, len) - Returns the leftmost `len`(`len` can be string type)
characters from the string `str`,if `len` is less or equal than 0 the result is
an empty string;
** right(str, len) - Returns the rightmost `len`(`len` can be string type)
characters from the string `str`,if `len` is less or equal than 0 the result is
an empty string.
They are different when `n`/`len` is negative. So maybe need to change Spark to
adapt to Postgres's meaning.
> Built-in udf left/right has inconsistent behavior
> -
>
> Key: SPARK-28036
> URL: https://issues.apache.org/jira/browse/SPARK-28036
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> PostgreSQL:
> {code:sql}
> postgres=# select left('ahoj', -2), right('ahoj', -2);
> left | right
> --+---
> ah | oj
> (1 row)
> {code}
> Spark SQL:
> {code:sql}
> spark-sql> select left('ahoj', -2), right('ahoj', -2);
> spark-sql>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-28036) Built-in udf left/right has inconsistent behavior
[
https://issues.apache.org/jira/browse/SPARK-28036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873696#comment-16873696
]
Shivu Sondur commented on SPARK-28036:
--
[~yumwang]
You can close this jira right?.
As it is working in spark.
Only difference is we should use offset without '-' symbol.
> Built-in udf left/right has inconsistent behavior
> -
>
> Key: SPARK-28036
> URL: https://issues.apache.org/jira/browse/SPARK-28036
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> PostgreSQL:
> {code:sql}
> postgres=# select left('ahoj', -2), right('ahoj', -2);
> left | right
> --+---
> ah | oj
> (1 row)
> {code}
> Spark SQL:
> {code:sql}
> spark-sql> select left('ahoj', -2), right('ahoj', -2);
> spark-sql>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-28036) Built-in udf left/right has inconsistent behavior
[
https://issues.apache.org/jira/browse/SPARK-28036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872050#comment-16872050
]
Yuming Wang commented on SPARK-28036:
-
{code:sql}
[root@spark-3267648 spark-3.0.0-SNAPSHOT-bin-3.2.0]# bin/spark-shell
19/06/24 23:10:12 WARN NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use
setLogLevel(newLevel).
Spark context Web UI available at http://spark-3267648.lvs02.dev.ebayc3.com:4040
Spark context available as 'sc' (master = local[*], app id =
local-1561443018277).
Spark session available as 'spark'.
Welcome to
__
/ __/__ ___ _/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 3.0.0-SNAPSHOT
/_/
Using Scala version 2.12.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_211)
Type in expressions to have them evaluated.
Type :help for more information.
scala> spark.sql("select left('ahoj', -2), right('ahoj', -2)").show
++-+
|left('ahoj', -2)|right('ahoj', -2)|
++-+
|| |
++-+
scala> spark.sql("select left('ahoj', 2), right('ahoj', 2)").show
+---++
|left('ahoj', 2)|right('ahoj', 2)|
+---++
| ah| oj|
+---++
{code}
> Built-in udf left/right has inconsistent behavior
> -
>
> Key: SPARK-28036
> URL: https://issues.apache.org/jira/browse/SPARK-28036
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> PostgreSQL:
> {code:sql}
> postgres=# select left('ahoj', -2), right('ahoj', -2);
> left | right
> --+---
> ah | oj
> (1 row)
> {code}
> Spark SQL:
> {code:sql}
> spark-sql> select left('ahoj', -2), right('ahoj', -2);
> spark-sql>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-28036) Built-in udf left/right has inconsistent behavior
[
https://issues.apache.org/jira/browse/SPARK-28036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872045#comment-16872045
]
Shivu Sondur commented on SPARK-28036:
--
[~yumwang]
select left('ahoj', 2), right('ahoj', 2);
use with '-' sign, it will work fine,
i tested in latest spark
> Built-in udf left/right has inconsistent behavior
> -
>
> Key: SPARK-28036
> URL: https://issues.apache.org/jira/browse/SPARK-28036
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
>Affects Versions: 3.0.0
>Reporter: Yuming Wang
>Priority: Major
>
> PostgreSQL:
> {code:sql}
> postgres=# select left('ahoj', -2), right('ahoj', -2);
> left | right
> --+---
> ah | oj
> (1 row)
> {code}
> Spark SQL:
> {code:sql}
> spark-sql> select left('ahoj', -2), right('ahoj', -2);
> spark-sql>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
