On the spark web UI, there are two tables - Running Applications and Running Drivers. What is the difference between these two?
When I submit an application to run on the cluster, I can see the driver table getting populated, but why is only one worker listed under Worker column? What is the role of this single worker? Shouldn't the driver program be running on the master node?