Re: Databricks Spark Parallelism and Shuffle Partitions

Hi Erica, On your cluster details, you can click on "Advanced", and then set those parameters in the "Spark" tab. Hope that helps. Thanks, Subhash On Thu, Feb 4, 2021 at 5:27 PM Erica Lin wrote: > Hello! > > Is there a way to set spark.sql.shuffle.partitions > and spark.default.parallelism in

Databricks Spark Parallelism and Shuffle Partitions

Hello! Is there a way to set spark.sql.shuffle.partitions and spark.default.parallelism in Databricks? I checked the event log and can't find those parameters in the log either. Is it something that Databricks sets automatically? Thanks, Erica