[
https://issues.apache.org/jira/browse/SPARK-5925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329303#comment-14329303
]
Laszlo Fesus commented on SPARK-5925:
-------------------------------------
Yes, but I thought it would be quite useful if the progress bar could be
"forwarded" from the more detailed view (spark web interface) to the master
yarn interface. That could fix the problem. Maybe this feature could be
implemented also for the _spark.yarn.historyServer.address_ functionality,
which actually does redirect us to the proper job details on the spark web
interface. (And would be even better, if we could retrieve this _updated_
progress bar also for the yarn restful interface)
> YARN - Spark progress bar stucks at 10% but after finishing shows 100%
> ----------------------------------------------------------------------
>
> Key: SPARK-5925
> URL: https://issues.apache.org/jira/browse/SPARK-5925
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 1.2.1
> Reporter: Laszlo Fesus
> Priority: Minor
>
> I did set up a yarn cluster (CDH5) and spark (1.2.1), and also started Spark
> History Server. Now I am able to click on more details on yarn's web
> interface and get redirected to the appropriate spark logs during both job
> execution and also after the job has finished.
> My only concern is that while a spark job is being executed (either
> yarn-client or yarn-cluster), the progress bar stucks at 10% and doesn't
> increase as for MapReduce jobs. After finishing, it shows 100% properly, but
> we are loosing the real-time tracking capability of the status bar.
> Also tested yarn restful web interface, and it retrieves again 10% during
> (yarn) spark job execution, and works well again after finishing. (I suppose
> for the while being I should have a look on Spark Job Server and see if it's
> possible to track the job via its restful web interface.)
> Did anyone else experience this behaviour? Thanks in advance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
