[jira] [Assigned] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-11 Thread Apache Spark (JIRA) 


 [ 
https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apache Spark reassigned SPARK-25690:


Assignee: Apache Spark

> Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied 
> infinitely
> ---
>
> Key: SPARK-25690
> URL: https://issues.apache.org/jira/browse/SPARK-25690
> Project: Spark
>  Issue Type: Sub-task
>  Components: Spark Core, SQL
>Affects Versions: 2.4.0
>Reporter: Maryann Xue
>Assignee: Apache Spark
>Priority: Major
>
> This was fixed in SPARK-24891 and was then broken by SPARK-25044.
> The unit test in {{AnalysisSuite}} added in SPARK-24891 should have failed 
> but didn't because it wasn't properly updated after the {{ScalaUDF}} 
> constructor signature change. In the meantime, the other two end-to-end tests 
> added in SPARK-24891 were shadowed by SPARK-24865.
> So the unit test mentioned above, if updated properly, can reproduce this 
> issue:
> {code:java}
> test("SPARK-24891 Fix HandleNullInputsForUDF rule") {
>   val a = testRelation.output(0)
>   val func = (x: Int, y: Int) => x + y
>   val udf1 = ScalaUDF(func, IntegerType, a :: a :: Nil, nullableTypes = false 
> :: false :: Nil)
>   val udf2 = ScalaUDF(func, IntegerType, a :: udf1 :: Nil, nullableTypes = 
> false :: false :: Nil)
>   val plan = Project(Alias(udf2, "")() :: Nil, testRelation)
>   comparePlans(plan.analyze, plan.analyze.analyze)
> }{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Assigned] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-11 Thread Apache Spark (JIRA) 


 [ 
https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apache Spark reassigned SPARK-25690:


Assignee: (was: Apache Spark)

> Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied 
> infinitely
> ---
>
> Key: SPARK-25690
> URL: https://issues.apache.org/jira/browse/SPARK-25690
> Project: Spark
>  Issue Type: Sub-task
>  Components: Spark Core, SQL
>Affects Versions: 2.4.0
>Reporter: Maryann Xue
>Priority: Major
>
> This was fixed in SPARK-24891 and was then broken by SPARK-25044.
> The unit test in {{AnalysisSuite}} added in SPARK-24891 should have failed 
> but didn't because it wasn't properly updated after the {{ScalaUDF}} 
> constructor signature change. In the meantime, the other two end-to-end tests 
> added in SPARK-24891 were shadowed by SPARK-24865.
> So the unit test mentioned above, if updated properly, can reproduce this 
> issue:
> {code:java}
> test("SPARK-24891 Fix HandleNullInputsForUDF rule") {
>   val a = testRelation.output(0)
>   val func = (x: Int, y: Int) => x + y
>   val udf1 = ScalaUDF(func, IntegerType, a :: a :: Nil, nullableTypes = false 
> :: false :: Nil)
>   val udf2 = ScalaUDF(func, IntegerType, a :: udf1 :: Nil, nullableTypes = 
> false :: false :: Nil)
>   val plan = Project(Alias(udf2, "")() :: Nil, testRelation)
>   comparePlans(plan.analyze, plan.analyze.analyze)
> }{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Assigned] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-09 Thread Sean Owen (JIRA) 


 [ 
https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Owen reassigned SPARK-25690:
-

 Assignee: (was: Sean Owen)
Fix Version/s: (was: 2.4.0)

> Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied 
> infinitely
> ---
>
> Key: SPARK-25690
> URL: https://issues.apache.org/jira/browse/SPARK-25690
> Project: Spark
>  Issue Type: Sub-task
>  Components: Spark Core, SQL
>Affects Versions: 2.4.0
>Reporter: Maryann Xue
>Priority: Major
>
> This was fixed in SPARK-24891 and was then broken by SPARK-25044.
> The tests added in SPARK-24891 were not good enough and the expected failures 
> were shadowed by SPARK-24865. For more details, please refer to SPARK-25650. 
> Code changes and tests in 
> [https://github.com/apache/spark/pull/22060/files#diff-f70523b948b7af21abddfa3ab7e1d7d6R72]
>  can help reproduce the issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org