Binding spark workers to a network interface

Hi all, Is there a configuration to force spark to use a specific network interface to communicate. The machines we are using have three network interfaces and we would like to bind them to a specific network interface. Best, Supun..

Re: Sorting tuples with byte key and byte value

ould > rather use Dataframe sort operation if performance is key. > > Regards, > Keith. > > http://keith-chapman.com > > > On Mon, Jul 15, 2019 at 8:45 AM Supun Kamburugamuve < > supun.kamburugam...@gmail.com> wrote: > >> Hi all, >> >> We

Sorting tuples with byte key and byte value

Hi all, We are trying to measure the sorting performance of Spark. We have a 16 node cluster with 48 cores and 256GB of ram in each machine and 10Gbps network. Let's say we are running with 128 parallel tasks and each partition generates about 1GB of data (total 128GB). We are using the method