I'm not sure if this is the case, but basic monitoring is described here:
https://spark.apache.org/docs/latest/monitoring.html
If it comes to something more sophisticated I was for example able to save
some messages into local logs and view them in YARN UI via http by editing
spark source code (use
Hi there,
I'm trying to run JavaSparkPi example on YARN with master = yarn-client but
I have a problem.
It runs smoothly with submitting application, first container for
Application Master works too.
When job is starting and there are some tasks to do I'm getting this warning
on console (I'm us
