Hi Martin, I tested the local mode in Spark on Rapids Accelerator and it works fine for me. The only possible issue is the CUDA 11.2 however the supported CUDA version as per https://nvidia.github.io/spark-rapids/docs/download.html is 11.0.
Here is a quick test using Spark local mode. Note: When I was testing this local mode, I make sure there is nothing in spark-defaults.conf so everything is clean. ====== scala> val df = sc.makeRDD(1 to 100, 6).toDF df: org.apache.spark.sql.DataFrame = [value: int] scala> val df2 = sc.makeRDD(1 to 100, 6).toDF df2: org.apache.spark.sql.DataFrame = [value: int] scala> df.select( $"value" as "a").join(df2.select($"value" as "b"), $"a" === $"b").count res0: Long = 100 scala> df.select( $"value" as "a").join(df2.select($"value" as "b"), $"a" === $"b").explain() == Physical Plan == GpuColumnarToRow false +- GpuShuffledHashJoin [a#29], [b#31], Inner, GpuBuildRight, false :- GpuShuffleCoalesce 2147483647 : +- GpuColumnarExchange gpuhashpartitioning(a#29, 10), ENSURE_REQUIREMENTS, [id=#221] : +- GpuProject [value#2 AS a#29] : +- GpuRowToColumnar TargetSize(2147483647) : +- *(1) SerializeFromObject [input[0, int, false] AS value#2] : +- Scan[obj#1] +- GpuCoalesceBatches RequireSingleBatch +- GpuShuffleCoalesce 2147483647 +- GpuColumnarExchange gpuhashpartitioning(b#31, 10), ENSURE_REQUIREMENTS, [id=#228] +- GpuProject [value#8 AS b#31] +- GpuRowToColumnar TargetSize(2147483647) +- *(2) SerializeFromObject [input[0, int, false] AS value#8] +- Scan[obj#7] ====== Thanks, Hao -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org