[
https://issues.apache.org/jira/browse/SPARK-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-5925:
-----------------------------
Target Version/s: (was: 1.2.1)
Issue Type: Improvement (was: Bug)
I don't know that it's as possible to know how long the entire execution will
take in a Spark job, in a way that you would estimate it for MapReduce.
> YARN - Spark progress bar stucks at 10% but after finishing shows 100%
> ----------------------------------------------------------------------
>
> Key: SPARK-5925
> URL: https://issues.apache.org/jira/browse/SPARK-5925
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 1.2.1
> Reporter: Laszlo Fesus
> Priority: Minor
>
> I did set up a yarn cluster (CDH5) and spark (1.2.1), and also started Spark
> History Server. Now I am able to click on more details on yarn's web
> interface and get redirected to the appropriate spark logs during both job
> execution and also after the job has finished.
> My only concern is that while a spark job is being executed (either
> yarn-client or yarn-cluster), the progress bar stucks at 10% and doesn't
> increase as for MapReduce jobs. After finishing, it shows 100% properly, but
> we are loosing the real-time tracking capability of the status bar.
> Also tested yarn restful web interface, and it retrieves again 10% during
> (yarn) spark job execution, and works well again after finishing. (I suppose
> for the while being I should have a look on Spark Job Server and see if it's
> possible to track the job via its restful web interface.)
> Did anyone else experience this behaviour? Thanks in advance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
