spark.yarn.am.memory 8g
spark.yarn.am.cores 4
spark.executor.memory 8g
spark.executor.cores 4
spark.yarn.executor.memoryOverhead 6144
spark.memory.offHeap.enabled true
spark.memory.offHeap.size 40
Best & Regards
Cyanny LIANG
. how many receivers will be start on a worker? We find that,
if the application got 5 executors, 10 receivers will be started on these
executors by random. Is that right?
--
Cyanny LIANG
worker subprocess, what is the memory limit
for each python worker? does spark.python.worker.memory affect the memory
of a python worker?
--
Best & Regards
Cyanny LIANG
email: lgrcya...@gmail.com
