Hi Marcelo,
I found the problem from
http://mail-archives.apache.org/mod_mbox/spark-user/201409.mbox/%3cCAL+LEBfzzjugOoB2iFFdz_=9TQsH=DaiKY=cvydfydg3ac5...@mail.gmail.com%3e
this link. The problem is the application I am running, is not generating
"APPLICATION_COMPLETE" file. If I add this file ma
Yes. I do see files, actually I missed copying the other settings:
spark.master spark://
skarri-lt05.redmond.corp.microsoft.com:7077
spark.eventLog.enabled true
spark.rdd.compress true
spark.storage.memoryFraction 1
spark.core.connection.ack.wait.timeout 6000
spark.ak
On Wed, Mar 4, 2015 at 10:08 AM, Srini Karri wrote:
> spark.executor.extraClassPath
> D:\\Apache\\spark-1.2.1-bin-hadoop2\\spark-1.2.1-bin-hadoop2.4\\bin\\classes
> spark.eventLog.dir
> D:/Apache/spark-1.2.1-bin-hadoop2/spark-1.2.1-bin-hadoop2.4/bin/tmp/spark-events
> spark.history.fs.logDirectory
Hi Todd and Marcelo,
Thanks for helping me. I was to able to lunch the history server on windows
with out any issues. One problem I am running into right now. I always get
the message no completed applications found in history server UI. But I was
able to browse through these applications from Spa
Spark applications shown in the RM's UI should have an "Application
Master" link when they're running. That takes you to the Spark UI for
that application where you can see all the information you're looking
for.
If you're running a history server and add
"spark.yarn.historyServer.address" to your
Hi Srini,
If you start the $SPARK_HOME/sbin/start-history-server, you should be able
to see the basic spark ui. You will not see the master, but you will be
able to see the rest as I recall. You also need to add an entry into the
spark-defaults.conf, something like this:
*## Make sure the host
Hi All,
I am having trouble finding data related to my requirement. Here is the
context, I have tried Standalone Spark Installation on Windows, I am able
to submit the logs, able to see the history of events. My question is, is
it possible to achieve the same monitoring UI experience with Yarn Clu
