[ https://issues.apache.org/jira/browse/SPARK-33045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235632#comment-17235632 ]
Apache Spark commented on SPARK-33045: -------------------------------------- User 'sunchao' has created a pull request for this issue: https://github.com/apache/spark/pull/30431 > Implement built-in LIKE ANY and LIKE ALL UDF > -------------------------------------------- > > Key: SPARK-33045 > URL: https://issues.apache.org/jira/browse/SPARK-33045 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.1.0 > Reporter: Yuming Wang > Assignee: jiaan.geng > Priority: Major > Fix For: 3.1.0 > > > We already support LIKE ANY / SOME / ALL syntax, but it will throw > {{StackOverflowError}} if there are many elements(more than 14378 elements). > We should implement built-in LIKE ANY and LIKE ALL UDF to fix this issue. > {noformat} > java.lang.StackOverflowError > at > scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36) > at > scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33) > at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38) > at scala.collection.generic.Growable.$plus$plus$eq(Growable.scala:62) > at scala.collection.generic.Growable.$plus$plus$eq$(Growable.scala:53) > at > scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:184) > at > scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:47) > at > scala.collection.generic.GenericCompanion.apply(GenericCompanion.scala:53) > at > org.apache.spark.sql.catalyst.expressions.BinaryExpression.children(Expression.scala:549) > at > org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:175) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1(TreeNode.scala:175) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1$adapted(TreeNode.scala:175) > at scala.collection.immutable.List.foreach(List.scala:392) > at > org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:175) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1(TreeNode.scala:175) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$foreach$1$adapted(TreeNode.scala:175) > at scala.collection.immutable.List.foreach(List.scala:392) > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org