ulysses you created SPARK-34396:
-----------------------------------
Summary: Add a new build-in function delegate
Key: SPARK-34396
URL: https://issues.apache.org/jira/browse/SPARK-34396
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.2.0
Reporter: ulysses you
Delegate is just like a big box that can include some other expressions. It
will execute all of children and return the last child result as its result.
The origin idea is from debug. Debug SQL is hard since SQL is always quite long
and complex. This new function can help debug with inject some help functions,
e.g., `println, sleep, raise_error`.
Two usage examples:
{code:java}
-- raw sql
INSERT INTO TABLE t1
SELECT coalesce(c1, c2) as c FROM t2
-- print the column data
INSERT INTO TABLE t1
SELECT delegate(
java_method('scala.Console', 'println', concat('c1: ', c1, ', c2: ', c2)),
coalesce(c1, c2)
) as c FROM t2{code}
{code:java}
-- raw sql
SELECT if(spark_partition_id() = 1, c1, raise_error('test error')) FROM t2
-- add a sleep time before throw error
SELECT if(spark_partition_id() = 1, c1, delegate(
java_method('java.lang.Thread', 'sleep', 3000l),
raise_error('test error')
)) FROM t2{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
