RussellSpitzer commented on issue #2533: URL: https://github.com/apache/iceberg/issues/2533#issuecomment-834876505
My guess is that something in our code is invoking a ZippedRDD (probably a broadcast join?) which ends up having 0 data in it. This is a ZippedRDD with 0 partitions, but there is an assertion error that you can't have such an RDD when we call .partitions https://github.com/apache/spark/blob/v3.0.2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2Exec.scala#L366 I'm going to see if I can figure out where we generate this RDD, basically we need to not have our underlying query.execute produce a zipped rdd if it is empty. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
