[
https://issues.apache.org/jira/browse/SPARK-10927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Baptiste Onofré updated SPARK-10927:
-----------------------------------------
Description:
Setting spark.eventLog.enabled to true, and a folder location for
spark.eventLog.dir provides the history UI for completed jobs.
It works fine for jobs without arguments, but if the job expects some arguments
(like JavaWordCount which expects the source file location), the UI is not
possible to provide application details:
{code}
Application history not found (app-20151005185136-0002)
No event logs found for application JavaWordCount in file:/tmp/spark. Did you
specify the correct logging directory?
{code}
However, in /tmp/spark, the file app-20151005185136-0002 is there. It seems
that the UI uses the application name (JavaWordCount) instead of the
application ID (app-20151005185136-0002) to get history details.
I will work on a fix around that.
was:
Setting spark.eventLog.enabled to true, and a folder location for
spark.eventLog.dir provides the history UI for completed jobs.
It works fine for jobs without arguments, but if the job expects some arguments
(like JavaWordCount which expects the source file location), the UI is not
possible to provide application details:
{code}
Application history not found (app-20151005185136-0002)
No event logs found for application JavaWordCount in file:/tmp/spark. Did you
specify the correct logging directory?
{/code}
However, in /tmp/spark, the file app-20151005185136-0002 is there. It seems
that the UI uses the application name (JavaWordCount) instead of the
application ID (app-20151005185136-0002) to get history details.
I will work on a fix around that.
> Spark history uses the application name instead of the ID
> ---------------------------------------------------------
>
> Key: SPARK-10927
> URL: https://issues.apache.org/jira/browse/SPARK-10927
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.5.1
> Reporter: Jean-Baptiste Onofré
>
> Setting spark.eventLog.enabled to true, and a folder location for
> spark.eventLog.dir provides the history UI for completed jobs.
> It works fine for jobs without arguments, but if the job expects some
> arguments (like JavaWordCount which expects the source file location), the UI
> is not possible to provide application details:
> {code}
> Application history not found (app-20151005185136-0002)
> No event logs found for application JavaWordCount in file:/tmp/spark. Did you
> specify the correct logging directory?
> {code}
> However, in /tmp/spark, the file app-20151005185136-0002 is there. It seems
> that the UI uses the application name (JavaWordCount) instead of the
> application ID (app-20151005185136-0002) to get history details.
> I will work on a fix around that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
