Hi Mich, Community - Do I need to specify it in the properties file in my
spark-submit ?
From: Mich Talebzadeh [mailto:m...@peridale.co.uk]
Sent: Thursday, February 18, 2016 4:28 PM
To: Sutanu Das; user@spark.apache.org
Subject: RE: Spark History Server NOT showing Jobs with Hortonworks
The jobs are normally shown under :4040/jobs/ in a normal set up not
using any vendor's flavoiur
Dr Mich Talebzadeh
LinkedIn
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
http://talebzadehmich.wordpress.com<http://talebzadehmich.wordpress.com/>
NOTE: The information in this email is proprietary and confidential. This
message is for the designated recipient only, if you are not the intended
recipient, you should destroy it immediately. Any information in this message
shall not be understood as given or endorsed by Peridale Technology Ltd, its
subsidiaries or their employees, unless expressly so stated. It is the
responsibility of the recipient to ensure that this email is virus free,
therefore neither Peridale Technology Ltd, its subsidiaries nor their employees
accept any responsibility.
From: Sutanu Das [mailto:sd2...@att.com]
Sent: 18 February 2016 22:22
To: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Spark History Server NOT showing Jobs with Hortonworks
Hi Community,
Challenged with Spark issues with Hortonworks (HDP 2.3.2_Spark 1.4.1) - The
Spark History Server is NOT showing the Spark Running Jobs in Local Mode
The local-host:4040/app/v1 is ALSO not working
How can I look at my local Spark job?
# Generated by Apache Ambari. Fri Feb 5 00:37:06 2016
spark.history.kerberos.keytab none
spark.history.kerberos.principal none
spark.history.provider org.apache.spark.deploy.yarn.history.YarnHistoryProvider
spark.history.ui.port 18080
spark.yarn.containerLauncherMaxThreads 25
spark.yarn.driver.memoryOverhead 2048
spark.yarn.executor.memoryOverhead 2048
spark.yarn.historyServer.address has-dal-0001.corp.wayport.net:18080
spark.yarn.max.executor.failures 3
spark.yarn.preserve.staging.files false
spark.yarn.queue default
spark.yarn.scheduler.heartbeat.interval-ms 5000
spark.yarn.services org.apache.spark.deploy.yarn.history.YarnHistoryService
spark.yarn.submit.file.replication 3
History Server
* Timeline Service Location: http://has-dal-0002.corp.wayport.net:8188/
* Last Updated: Feb 18, 2016 10:09:12 PM UTC
* Service Started: Feb 5, 2016 12:37:15 AM UTC
* Current Time: Feb 18, 2016 10:10:46 PM UTC
* Timeline Service: Timeline service is enabled
* History Provider: Apache Hadoop YARN Timeline Service