Hi, Say we have 4 nodes with 2 cores each in stand alone mode. I'd like to dedicate 4 cores to a streaming application. I can do this via spark submit by:
spark-submit .... --total-executor-cores 4 However, this assigns one core per machine. I would like to use 2 cores on 2 machines instead, leaving the other two machines untouched. Is this possible? Is there a downside to doing this? My thinking is that I should be able to reduce quite a bit of network traffic if all machines are not involved. Thanks, Ashic.