Hi all: i want to ask a question about the metrics to show the executor is fully used the memory. in the log i always saw following in the log, i guess this means i did not fully use the executor 's memory. but i don't want to open the log to view, is there any metrics to show it?
my config: SPARK_EXECUTOR_MEMORY==16G SPARK_EXECUTOR_CORE=2 20/05/15 02:39:01 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on lvshdc5dn3718.lvs.paypalinc.com:45488 (size: 42.7 KB, free: 4.1 GB) 20/05/15 02:39:01 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on lvshdc5dn4353.lvs.paypalinc.com:35598 (size: 42.7 KB, free: 4.1 GB) 20/05/15 02:39:01 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on lvshdc5dn3847.lvs.paypalinc.com:35152 (size: 42.7 KB, free: 4.1 GB) my question is "is there any mentrics to show the usage of executor on memory or CPU" , appreciate if you have any suggestion Best Regards Kelly Zhang
