I tried to access worker info from spark context but it seems spark context does no expose such API. The reason of doing that is: it seems spark context itself does not have logic to detect if its workers are in dead status. So I like to add such logic by myself.
BTW, it seems spark web UI has some logic of detecting dead workers. But all relevant classes are declared as private for the spark package scope. Please let me know how to solve this issue (or if there is an alternative way to achieve the same purpose) Thanks