Adam Breindel created SPARK-17896:

             Summary: Dataset groupByKey + reduceGroups fails with 
codegen-related exception
                 Key: SPARK-17896
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.0.1
         Environment: Databricks, MacOS
            Reporter: Adam Breindel

possible regression: works on 2.0, fails on 2.0.1
following code raises exception related to wholestage codegen:

case class Zip(city:String, zip:String, state:String)

val z1 = Zip("New York", "10000", "NY")
val z2 = Zip("New York", "10001", "NY")
val z3 = Zip("Chicago", "60606", "IL")

val zips = sc.parallelize(Seq(z1, z2, z3)).toDS

zips.groupByKey(_.state).reduceGroups((z1, z2) => Zip("*", + " " +, z1.state)).show

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to