Hi, I run spark streaming job in cluster mode. This means that driver can run in any data node. And Spark UI can run in any dynamic port. At present, I know about the port by looking at container logs that look something like this -
server.AbstractConnector: Started SelectChannelConnector@0.0.0.0:50571 INFO util.Utils: Successfully started service 'SparkUI' on port 50571. INFO ui.SparkUI: Started SparkUI at http://xxx:50571 Is there any way to know about the UI port automatically using some API? Thanks Ashish
