Hi, These two threads that you sent seem to be duplicates of each other?
Anyhow I trust that you are familiar with the concept of shuffle in Spark. Spark Shuffle is an expensive operation since it involves the following - Disk I/O - Involves data serialization and deserialization - Network I/O Basically these are based on the concept of map/reduce in Spark and these parameters you posted relate to various aspects of threading and concurrency. HTH Mich Talebzadeh, Solutions Architect/Engineering Lead London United Kingdom view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Fri, 18 Aug 2023 at 20:39, Nebi Aydin <nayd...@binghamton.edu.invalid> wrote: > > I want to learn differences among below thread configurations. > > spark.shuffle.io.serverThreads > spark.shuffle.io.clientThreads > spark.shuffle.io.threads > spark.rpc.io.serverThreads > spark.rpc.io.clientThreads > spark.rpc.io.threads > > Thanks. >
