[ https://issues.apache.org/jira/browse/SPARK-32952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575252#comment-17575252 ]
Vivian Kong commented on SPARK-32952: ------------------------------------- The test is still failing on Spark v3.3.0 on IBM Z. We appreciate any pointers community can share so we can look into it. Thanks. > Test failure on IBM Z: CoalesceShufflePartitionsSuite: - determining the > number of reducers: complex query 1 > ------------------------------------------------------------------------------------------------------------ > > Key: SPARK-32952 > URL: https://issues.apache.org/jira/browse/SPARK-32952 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.0.1 > Environment: Linux on IBM Z (s390x). > Reporter: Michael Munday > Priority: Minor > Labels: big-endian > > I'm seeing the test 'CoalesceShufflePartitionsSuite: - determining the number > of reducers: complex query 1' fail on IBM Z with the wrong number of > partitions (1 instead of 2). It's strange because none of the other tests > fail. > I'd be grateful for any hints as to how the number of partitions is > calculated. Could that calculation be affected by incorrect unsafe code? Is > there a way to trace the calculation? > {noformat} > CoalesceShufflePartitionsSuite: > - determining the number of reducers: aggregate > operator(minNumPostShufflePartitions: 5) > - determining the number of reducers: join > operator(minNumPostShufflePartitions: 5) > - determining the number of reducers: complex query > 1(minNumPostShufflePartitions: 5) > - determining the number of reducers: complex query > 2(minNumPostShufflePartitions: 5) > - determining the number of reducers: plan already > partitioned(minNumPostShufflePartitions: 5) > - determining the number of reducers: aggregate operator > - determining the number of reducers: join operator > - determining the number of reducers: complex query 1 *** FAILED *** > 1 did not equal 2 (CoalesceShufflePartitionsSuite.scala:221) > - determining the number of reducers: complex query 2 > - determining the number of reducers: plan already partitioned > - SPARK-24705 adaptive query execution works correctly when exchange reuse > enabled > - Do not reduce the number of shuffle partition for repartition > - Union two datasets with different pre-shuffle partition number{noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org