Re: Viewing web UI after fact
>>>> Is there any way view history of applications statistics in master ui >>>>>> after restarting master server? I have all logs ing /tmp/spark-events/ >>>>>> but >>>>>> when I start history server in this directory it says "No Completed >>>>>> Applications Found". Maybe I could copy this logs to dir used by master >>>>>> server but I couldn't find any. Or maybe I'm doing something wrong >>>>>> launching history server. >>>>>> Do you have any idea how to solve it? >>>>>> >>>>>> Thanks, >>>>>> Grzegorz >>>>>> >>>>>> >>>>>> On Thu, Aug 14, 2014 at 10:53 AM, Grzegorz Białek < >>>>>> grzegorz.bia...@codilime.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Thank you both for your answers. Browsing using Master UI works >>>>>>> fine. Unfortunately History Server shows "No Completed Applications >>>>>>> Found" >>>>>>> even if logs exists under given directory, but using Master UI is enough >>>>>>> for me. >>>>>>> >>>>>>> Best regards, >>>>>>> Grzegorz >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Aug 13, 2014 at 8:09 PM, Andrew Or >>>>>>> wrote: >>>>>>> >>>>>>>> The Spark UI isn't available through the same address; otherwise >>>>>>>> new applications won't be able to bind to it. Once the old application >>>>>>>> finishes, the standalone Master renders the after-the-fact application >>>>>>>> UI >>>>>>>> and exposes it under a different URL. To see this, go to the Master UI >>>>>>>> (:8080) and click on your application in the "Completed >>>>>>>> Applications" table. >>>>>>>> >>>>>>>> >>>>>>>> 2014-08-13 10:56 GMT-07:00 Matei Zaharia : >>>>>>>> >>>>>>>> Take a look at http://spark.apache.org/docs/latest/monitoring.html >>>>>>>>> -- you need to launch a history server to serve the logs. >>>>>>>>> >>>>>>>>> Matei >>>>>>>>> >>>>>>>>> On August 13, 2014 at 2:03:08 AM, grzegorz-bialek ( >>>>>>>>> grzegorz.bia...@codilime.com) wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> I wanted to access Spark web UI after application stops. I set >>>>>>>>> spark.eventLog.enabled to true and logs are availaible >>>>>>>>> in JSON format in /tmp/spark-event but web UI isn't available >>>>>>>>> under address >>>>>>>>> http://:4040 >>>>>>>>> I'm running Spark in standalone mode. >>>>>>>>> >>>>>>>>> What should I do to access web UI after application ends? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Grzegorz >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> View this message in context: >>>>>>>>> http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html >>>>>>>>> Sent from the Apache Spark User List mailing list archive at >>>>>>>>> Nabble.com. >>>>>>>>> >>>>>>>>> - >>>>>>>>> >>>>>>>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>>>>>>> For additional commands, e-mail: user-h...@spark.apache.org >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > -- *Best Wishes!* *Li Hu(李浒) | Graduate Student* *Institute for Interdisciplinary Information Sciences(IIIS <http://iiis.tsinghua.edu.cn/>)* *Tsinghua University, China* *Email: lihu...@gmail.com * *Homepage: http://iiis.tsinghua.edu.cn/zh/lihu/ <http://iiis.tsinghua.edu.cn/zh/lihu/>*
Re: Viewing web UI after fact
rz.bia...@codilime.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Thank you both for your answers. Browsing using Master UI works fine. >>>>>> Unfortunately History Server shows "No Completed Applications Found" even >>>>>> if logs exists under given directory, but using Master UI is enough for >>>>>> me. >>>>>> >>>>>> Best regards, >>>>>> Grzegorz >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Aug 13, 2014 at 8:09 PM, Andrew Or >>>>>> wrote: >>>>>> >>>>>>> The Spark UI isn't available through the same address; otherwise new >>>>>>> applications won't be able to bind to it. Once the old application >>>>>>> finishes, the standalone Master renders the after-the-fact application >>>>>>> UI >>>>>>> and exposes it under a different URL. To see this, go to the Master UI >>>>>>> (:8080) and click on your application in the "Completed >>>>>>> Applications" table. >>>>>>> >>>>>>> >>>>>>> 2014-08-13 10:56 GMT-07:00 Matei Zaharia : >>>>>>> >>>>>>> Take a look at http://spark.apache.org/docs/latest/monitoring.html >>>>>>>> -- you need to launch a history server to serve the logs. >>>>>>>> >>>>>>>> Matei >>>>>>>> >>>>>>>> On August 13, 2014 at 2:03:08 AM, grzegorz-bialek ( >>>>>>>> grzegorz.bia...@codilime.com) wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> I wanted to access Spark web UI after application stops. I set >>>>>>>> spark.eventLog.enabled to true and logs are availaible >>>>>>>> in JSON format in /tmp/spark-event but web UI isn't available under >>>>>>>> address >>>>>>>> http://:4040 >>>>>>>> I'm running Spark in standalone mode. >>>>>>>> >>>>>>>> What should I do to access web UI after application ends? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Grzegorz >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> View this message in context: >>>>>>>> http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html >>>>>>>> Sent from the Apache Spark User List mailing list archive at >>>>>>>> Nabble.com. >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>>>>>> For additional commands, e-mail: user-h...@spark.apache.org >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
Re: Viewing web UI after fact
; Thanks, >>>>> Grzegorz >>>>> >>>>> >>>>> On Thu, Aug 14, 2014 at 10:53 AM, Grzegorz Białek < >>>>> grzegorz.bia...@codilime.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Thank you both for your answers. Browsing using Master UI works fine. >>>>>> Unfortunately History Server shows "No Completed Applications Found" even >>>>>> if logs exists under given directory, but using Master UI is enough for >>>>>> me. >>>>>> >>>>>> Best regards, >>>>>> Grzegorz >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Aug 13, 2014 at 8:09 PM, Andrew Or >>>>>> wrote: >>>>>> >>>>>>> The Spark UI isn't available through the same address; otherwise new >>>>>>> applications won't be able to bind to it. Once the old application >>>>>>> finishes, the standalone Master renders the after-the-fact application >>>>>>> UI >>>>>>> and exposes it under a different URL. To see this, go to the Master UI >>>>>>> (:8080) and click on your application in the "Completed >>>>>>> Applications" table. >>>>>>> >>>>>>> >>>>>>> 2014-08-13 10:56 GMT-07:00 Matei Zaharia : >>>>>>> >>>>>>> Take a look at http://spark.apache.org/docs/latest/monitoring.html >>>>>>>> -- you need to launch a history server to serve the logs. >>>>>>>> >>>>>>>> Matei >>>>>>>> >>>>>>>> On August 13, 2014 at 2:03:08 AM, grzegorz-bialek ( >>>>>>>> grzegorz.bia...@codilime.com) wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> I wanted to access Spark web UI after application stops. I set >>>>>>>> spark.eventLog.enabled to true and logs are availaible >>>>>>>> in JSON format in /tmp/spark-event but web UI isn't available under >>>>>>>> address >>>>>>>> http://:4040 >>>>>>>> I'm running Spark in standalone mode. >>>>>>>> >>>>>>>> What should I do to access web UI after application ends? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Grzegorz >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> View this message in context: >>>>>>>> http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html >>>>>>>> Sent from the Apache Spark User List mailing list archive at >>>>>>>> Nabble.com. >>>>>>>> >>>>>>>> - >>>>>>>> >>>>>>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>>>>>> For additional commands, e-mail: user-h...@spark.apache.org >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
Re: Viewing web UI after fact
The Spark UI isn't available through the same address; otherwise new >>>>>> applications won't be able to bind to it. Once the old application >>>>>> finishes, the standalone Master renders the after-the-fact application UI >>>>>> and exposes it under a different URL. To see this, go to the Master UI >>>>>> (:8080) and click on your application in the "Completed >>>>>> Applications" table. >>>>>> >>>>>> >>>>>> 2014-08-13 10:56 GMT-07:00 Matei Zaharia : >>>>>> >>>>>> Take a look at http://spark.apache.org/docs/latest/monitoring.html >>>>>>> -- you need to launch a history server to serve the logs. >>>>>>> >>>>>>> Matei >>>>>>> >>>>>>> On August 13, 2014 at 2:03:08 AM, grzegorz-bialek ( >>>>>>> grzegorz.bia...@codilime.com) wrote: >>>>>>> >>>>>>> Hi, >>>>>>> I wanted to access Spark web UI after application stops. I set >>>>>>> spark.eventLog.enabled to true and logs are availaible >>>>>>> in JSON format in /tmp/spark-event but web UI isn't available under >>>>>>> address >>>>>>> http://:4040 >>>>>>> I'm running Spark in standalone mode. >>>>>>> >>>>>>> What should I do to access web UI after application ends? >>>>>>> >>>>>>> Thanks, >>>>>>> Grzegorz >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> View this message in context: >>>>>>> http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html >>>>>>> Sent from the Apache Spark User List mailing list archive at >>>>>>> Nabble.com. >>>>>>> >>>>>>> - >>>>>>> >>>>>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>>>>> For additional commands, e-mail: user-h...@spark.apache.org >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
Re: Viewing web UI after fact
Hi Grzegorz, Can you verify that there are "APPLICATION_COMPLETE" files in the event log directories? E.g. Does file:/tmp/spark-events/app-name-1234567890/APPLICATION_COMPLETE exist? If not, it could be that your application didn't call sc.stop(), so the "ApplicationEnd" event is not actually logged. The HistoryServer looks for this special file to identify applications to display. You could also try manually adding the "APPLICATION_COMPLETE" file to this directory; the HistoryServer should pick this up and display the application, though the information displayed will be incomplete because the log did not capture all the events (sc.stop() does a final close() on the file written). Andrew 2014-09-05 1:50 GMT-07:00 Grzegorz Białek : > Hi Andrew, > > thank you very much for your answer. Unfortunately it still doesn't work. > I'm using Spark 1.0.0, and I start history server running > sbin/start-history-server.sh , although I also set > SPARK_HISTORY_OPTS=-Dspark.history.fs.logDirectory in conf/spark-env.sh. > I tried also other dir than /tmp/spark-events which have all possible > permissions enabled. Also adding file: (and file://) didn't help - history > server still shows: > History Server > Event Log Location: file:/tmp/spark-events/ > No Completed Applications Found. > > Best regards, > Grzegorz > > > On Thu, Sep 4, 2014 at 8:20 PM, Andrew Or wrote: > >> Hi Grzegorz, >> >> Sorry for the late response. Unfortunately, if the Master UI doesn't know >> about your applications (they are "completed" with respect to a different >> Master), then it can't regenerate the UIs even if the logs exist. You will >> have to use the history server for that. >> >> How did you start the history server? If you are using Spark <=1.0, you >> can pass the directory as an argument to the sbin/start-history-server.sh >> script. Otherwise, you may need to set the following in your >> conf/spark-env.sh to specify the log directory: >> >> export >> SPARK_HISTORY_OPTS=-Dspark.history.fs.logDirectory=/tmp/spark-events >> >> It could also be a permissions thing. Make sure your logs in >> /tmp/spark-events are accessible by the JVM that runs the history server. >> Also, there's a chance that "/tmp/spark-events" is interpreted as an HDFS >> path depending on which Spark version you're running. To resolve any >> ambiguity, you may set the log path to "file:/tmp/spark-events" instead. >> But first verify whether they actually exist. >> >> Let me know if you get it working, >> -Andrew >> >> >> >> 2014-08-19 8:23 GMT-07:00 Grzegorz Białek : >> >> Hi, >>> Is there any way view history of applications statistics in master ui >>> after restarting master server? I have all logs ing /tmp/spark-events/ but >>> when I start history server in this directory it says "No Completed >>> Applications Found". Maybe I could copy this logs to dir used by master >>> server but I couldn't find any. Or maybe I'm doing something wrong >>> launching history server. >>> Do you have any idea how to solve it? >>> >>> Thanks, >>> Grzegorz >>> >>> >>> On Thu, Aug 14, 2014 at 10:53 AM, Grzegorz Białek < >>> grzegorz.bia...@codilime.com> wrote: >>> >>>> Hi, >>>> >>>> Thank you both for your answers. Browsing using Master UI works fine. >>>> Unfortunately History Server shows "No Completed Applications Found" even >>>> if logs exists under given directory, but using Master UI is enough for me. >>>> >>>> Best regards, >>>> Grzegorz >>>> >>>> >>>> >>>> On Wed, Aug 13, 2014 at 8:09 PM, Andrew Or >>>> wrote: >>>> >>>>> The Spark UI isn't available through the same address; otherwise new >>>>> applications won't be able to bind to it. Once the old application >>>>> finishes, the standalone Master renders the after-the-fact application UI >>>>> and exposes it under a different URL. To see this, go to the Master UI >>>>> (:8080) and click on your application in the "Completed >>>>> Applications" table. >>>>> >>>>> >>>>> 2014-08-13 10:56 GMT-07:00 Matei Zaharia : >>>>> >>>>> Take a look at http://spark.apache.org/docs/latest/monitoring.ht
Re: Viewing web UI after fact
Hi Andrew, thank you very much for your answer. Unfortunately it still doesn't work. I'm using Spark 1.0.0, and I start history server running sbin/start-history-server.sh , although I also set SPARK_HISTORY_OPTS=-Dspark.history.fs.logDirectory in conf/spark-env.sh. I tried also other dir than /tmp/spark-events which have all possible permissions enabled. Also adding file: (and file://) didn't help - history server still shows: History Server Event Log Location: file:/tmp/spark-events/ No Completed Applications Found. Best regards, Grzegorz On Thu, Sep 4, 2014 at 8:20 PM, Andrew Or wrote: > Hi Grzegorz, > > Sorry for the late response. Unfortunately, if the Master UI doesn't know > about your applications (they are "completed" with respect to a different > Master), then it can't regenerate the UIs even if the logs exist. You will > have to use the history server for that. > > How did you start the history server? If you are using Spark <=1.0, you > can pass the directory as an argument to the sbin/start-history-server.sh > script. Otherwise, you may need to set the following in your > conf/spark-env.sh to specify the log directory: > > export SPARK_HISTORY_OPTS=-Dspark.history.fs.logDirectory=/tmp/spark-events > > It could also be a permissions thing. Make sure your logs in > /tmp/spark-events are accessible by the JVM that runs the history server. > Also, there's a chance that "/tmp/spark-events" is interpreted as an HDFS > path depending on which Spark version you're running. To resolve any > ambiguity, you may set the log path to "file:/tmp/spark-events" instead. > But first verify whether they actually exist. > > Let me know if you get it working, > -Andrew > > > > 2014-08-19 8:23 GMT-07:00 Grzegorz Białek : > > Hi, >> Is there any way view history of applications statistics in master ui >> after restarting master server? I have all logs ing /tmp/spark-events/ but >> when I start history server in this directory it says "No Completed >> Applications Found". Maybe I could copy this logs to dir used by master >> server but I couldn't find any. Or maybe I'm doing something wrong >> launching history server. >> Do you have any idea how to solve it? >> >> Thanks, >> Grzegorz >> >> >> On Thu, Aug 14, 2014 at 10:53 AM, Grzegorz Białek < >> grzegorz.bia...@codilime.com> wrote: >> >>> Hi, >>> >>> Thank you both for your answers. Browsing using Master UI works fine. >>> Unfortunately History Server shows "No Completed Applications Found" even >>> if logs exists under given directory, but using Master UI is enough for me. >>> >>> Best regards, >>> Grzegorz >>> >>> >>> >>> On Wed, Aug 13, 2014 at 8:09 PM, Andrew Or >>> wrote: >>> >>>> The Spark UI isn't available through the same address; otherwise new >>>> applications won't be able to bind to it. Once the old application >>>> finishes, the standalone Master renders the after-the-fact application UI >>>> and exposes it under a different URL. To see this, go to the Master UI >>>> (:8080) and click on your application in the "Completed >>>> Applications" table. >>>> >>>> >>>> 2014-08-13 10:56 GMT-07:00 Matei Zaharia : >>>> >>>> Take a look at http://spark.apache.org/docs/latest/monitoring.html -- >>>>> you need to launch a history server to serve the logs. >>>>> >>>>> Matei >>>>> >>>>> On August 13, 2014 at 2:03:08 AM, grzegorz-bialek ( >>>>> grzegorz.bia...@codilime.com) wrote: >>>>> >>>>> Hi, >>>>> I wanted to access Spark web UI after application stops. I set >>>>> spark.eventLog.enabled to true and logs are availaible >>>>> in JSON format in /tmp/spark-event but web UI isn't available under >>>>> address >>>>> http://:4040 >>>>> I'm running Spark in standalone mode. >>>>> >>>>> What should I do to access web UI after application ends? >>>>> >>>>> Thanks, >>>>> Grzegorz >>>>> >>>>> >>>>> >>>>> -- >>>>> View this message in context: >>>>> http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html >>>>> Sent from the Apache Spark User List mailing list archive at >>>>> Nabble.com. >>>>> >>>>> - >>>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>>> For additional commands, e-mail: user-h...@spark.apache.org >>>>> >>>>> >>>> >>> >> >
Re: Viewing web UI after fact
Hi Grzegorz, Sorry for the late response. Unfortunately, if the Master UI doesn't know about your applications (they are "completed" with respect to a different Master), then it can't regenerate the UIs even if the logs exist. You will have to use the history server for that. How did you start the history server? If you are using Spark <=1.0, you can pass the directory as an argument to the sbin/start-history-server.sh script. Otherwise, you may need to set the following in your conf/spark-env.sh to specify the log directory: export SPARK_HISTORY_OPTS=-Dspark.history.fs.logDirectory=/tmp/spark-events It could also be a permissions thing. Make sure your logs in /tmp/spark-events are accessible by the JVM that runs the history server. Also, there's a chance that "/tmp/spark-events" is interpreted as an HDFS path depending on which Spark version you're running. To resolve any ambiguity, you may set the log path to "file:/tmp/spark-events" instead. But first verify whether they actually exist. Let me know if you get it working, -Andrew 2014-08-19 8:23 GMT-07:00 Grzegorz Białek : > Hi, > Is there any way view history of applications statistics in master ui > after restarting master server? I have all logs ing /tmp/spark-events/ but > when I start history server in this directory it says "No Completed > Applications Found". Maybe I could copy this logs to dir used by master > server but I couldn't find any. Or maybe I'm doing something wrong > launching history server. > Do you have any idea how to solve it? > > Thanks, > Grzegorz > > > On Thu, Aug 14, 2014 at 10:53 AM, Grzegorz Białek < > grzegorz.bia...@codilime.com> wrote: > >> Hi, >> >> Thank you both for your answers. Browsing using Master UI works fine. >> Unfortunately History Server shows "No Completed Applications Found" even >> if logs exists under given directory, but using Master UI is enough for me. >> >> Best regards, >> Grzegorz >> >> >> >> On Wed, Aug 13, 2014 at 8:09 PM, Andrew Or wrote: >> >>> The Spark UI isn't available through the same address; otherwise new >>> applications won't be able to bind to it. Once the old application >>> finishes, the standalone Master renders the after-the-fact application UI >>> and exposes it under a different URL. To see this, go to the Master UI >>> (:8080) and click on your application in the "Completed >>> Applications" table. >>> >>> >>> 2014-08-13 10:56 GMT-07:00 Matei Zaharia : >>> >>> Take a look at http://spark.apache.org/docs/latest/monitoring.html -- >>>> you need to launch a history server to serve the logs. >>>> >>>> Matei >>>> >>>> On August 13, 2014 at 2:03:08 AM, grzegorz-bialek ( >>>> grzegorz.bia...@codilime.com) wrote: >>>> >>>> Hi, >>>> I wanted to access Spark web UI after application stops. I set >>>> spark.eventLog.enabled to true and logs are availaible >>>> in JSON format in /tmp/spark-event but web UI isn't available under >>>> address >>>> http://:4040 >>>> I'm running Spark in standalone mode. >>>> >>>> What should I do to access web UI after application ends? >>>> >>>> Thanks, >>>> Grzegorz >>>> >>>> >>>> >>>> -- >>>> View this message in context: >>>> http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html >>>> Sent from the Apache Spark User List mailing list archive at >>>> Nabble.com. >>>> >>>> - >>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>> For additional commands, e-mail: user-h...@spark.apache.org >>>> >>>> >>> >> >
Re: Viewing web UI after fact
Hi, Is there any way view history of applications statistics in master ui after restarting master server? I have all logs ing /tmp/spark-events/ but when I start history server in this directory it says "No Completed Applications Found". Maybe I could copy this logs to dir used by master server but I couldn't find any. Or maybe I'm doing something wrong launching history server. Do you have any idea how to solve it? Thanks, Grzegorz On Thu, Aug 14, 2014 at 10:53 AM, Grzegorz Białek < grzegorz.bia...@codilime.com> wrote: > Hi, > > Thank you both for your answers. Browsing using Master UI works fine. > Unfortunately History Server shows "No Completed Applications Found" even > if logs exists under given directory, but using Master UI is enough for me. > > Best regards, > Grzegorz > > > > On Wed, Aug 13, 2014 at 8:09 PM, Andrew Or wrote: > >> The Spark UI isn't available through the same address; otherwise new >> applications won't be able to bind to it. Once the old application >> finishes, the standalone Master renders the after-the-fact application UI >> and exposes it under a different URL. To see this, go to the Master UI >> (:8080) and click on your application in the "Completed >> Applications" table. >> >> >> 2014-08-13 10:56 GMT-07:00 Matei Zaharia : >> >> Take a look at http://spark.apache.org/docs/latest/monitoring.html -- >>> you need to launch a history server to serve the logs. >>> >>> Matei >>> >>> On August 13, 2014 at 2:03:08 AM, grzegorz-bialek ( >>> grzegorz.bia...@codilime.com) wrote: >>> >>> Hi, >>> I wanted to access Spark web UI after application stops. I set >>> spark.eventLog.enabled to true and logs are availaible >>> in JSON format in /tmp/spark-event but web UI isn't available under >>> address >>> http://:4040 >>> I'm running Spark in standalone mode. >>> >>> What should I do to access web UI after application ends? >>> >>> Thanks, >>> Grzegorz >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html >>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>> >>> - >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>> For additional commands, e-mail: user-h...@spark.apache.org >>> >>> >> >
Re: Viewing web UI after fact
Hi, Thank you both for your answers. Browsing using Master UI works fine. Unfortunately History Server shows "No Completed Applications Found" even if logs exists under given directory, but using Master UI is enough for me. Best regards, Grzegorz On Wed, Aug 13, 2014 at 8:09 PM, Andrew Or wrote: > The Spark UI isn't available through the same address; otherwise new > applications won't be able to bind to it. Once the old application > finishes, the standalone Master renders the after-the-fact application UI > and exposes it under a different URL. To see this, go to the Master UI > (:8080) and click on your application in the "Completed > Applications" table. > > > 2014-08-13 10:56 GMT-07:00 Matei Zaharia : > > Take a look at http://spark.apache.org/docs/latest/monitoring.html -- you >> need to launch a history server to serve the logs. >> >> Matei >> >> On August 13, 2014 at 2:03:08 AM, grzegorz-bialek ( >> grzegorz.bia...@codilime.com) wrote: >> >> Hi, >> I wanted to access Spark web UI after application stops. I set >> spark.eventLog.enabled to true and logs are availaible >> in JSON format in /tmp/spark-event but web UI isn't available under >> address >> http://:4040 >> I'm running Spark in standalone mode. >> >> What should I do to access web UI after application ends? >> >> Thanks, >> Grzegorz >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> - >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> >
Re: Viewing web UI after fact
The Spark UI isn't available through the same address; otherwise new applications won't be able to bind to it. Once the old application finishes, the standalone Master renders the after-the-fact application UI and exposes it under a different URL. To see this, go to the Master UI (:8080) and click on your application in the "Completed Applications" table. 2014-08-13 10:56 GMT-07:00 Matei Zaharia : > Take a look at http://spark.apache.org/docs/latest/monitoring.html -- you > need to launch a history server to serve the logs. > > Matei > > On August 13, 2014 at 2:03:08 AM, grzegorz-bialek ( > grzegorz.bia...@codilime.com) wrote: > > Hi, > I wanted to access Spark web UI after application stops. I set > spark.eventLog.enabled to true and logs are availaible > in JSON format in /tmp/spark-event but web UI isn't available under > address > http://:4040 > I'm running Spark in standalone mode. > > What should I do to access web UI after application ends? > > Thanks, > Grzegorz > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
Re: Viewing web UI after fact
Take a look at http://spark.apache.org/docs/latest/monitoring.html -- you need to launch a history server to serve the logs. Matei On August 13, 2014 at 2:03:08 AM, grzegorz-bialek (grzegorz.bia...@codilime.com) wrote: Hi, I wanted to access Spark web UI after application stops. I set spark.eventLog.enabled to true and logs are availaible in JSON format in /tmp/spark-event but web UI isn't available under address http://:4040 I'm running Spark in standalone mode. What should I do to access web UI after application ends? Thanks, Grzegorz -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Viewing web UI after fact
Hi, I wanted to access Spark web UI after application stops. I set spark.eventLog.enabled to true and logs are availaible in JSON format in /tmp/spark-event but web UI isn't available under address http://:4040 I'm running Spark in standalone mode. What should I do to access web UI after application ends? Thanks, Grzegorz -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
