[jira] [Created] (SPARK-21376) Token is not renewed in yarn client process in cluster mode
Yesha Vora created SPARK-21376:
--
Summary: Token is not renewed in yarn client process in cluster
mode
Key: SPARK-21376
URL: https://issues.apache.org/jira/browse/SPARK-21376
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.1.0
Reporter: Yesha Vora
STR:
* Set below config in spark-default.conf
{code}
spark.yarn.security.credentials.hbase.enabled true
spark.hbase.connector.security.credentials.enabled false{code}
* Set below config in hdfs-site.xml
{code}
'dfs.namenode.delegation.token.max-lifetime':'4320'
'dfs.namenode.delegation.token.renew-interval':'2880' {code}
* Run HDFSWordcount streaming app in yarn-cluster mode for 25 hrs.
After 25 hours, noticing that HDFS Wordcount job is hitting
HDFS_DELEGATION_TOKEN renewal issue.
{code}
17/06/28 10:49:47 WARN Client: Exception encountered while connecting to the
server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 230 for hrt_qa) is expired
17/06/28 10:49:47 WARN Client: Failed to cleanup staging dir
hdfs://mycluster0/user/hrt_qa/.sparkStaging/application_1498539861056_0015
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 230 for hrt_qa) is expired
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1554)
at org.apache.hadoop.ipc.Client.call(Client.java:1498){code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-21377) Jars pulled from "--packages" are not added into AM classpath
[
https://issues.apache.org/jira/browse/SPARK-21377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-21377:
---
Description:
STR:
* Set below config in spark-default.conf
{code}
spark.yarn.security.credentials.hbase.enabled true
spark.hbase.connector.security.credentials.enabled false{code}
* Set below config in hdfs-site.xml
{code}
'dfs.namenode.delegation.token.max-lifetime':'4320'
'dfs.namenode.delegation.token.renew-interval':'2880' {code}
* Set below config in hbase-site.xml
{code}
'hbase.auth.token.max.lifetime': '2880' {code}
* Run an application with SHC package
{code}
spark-submit --class
org.apache.spark.sql.execution.datasources.hbase.examples.LRJobForDataSources
--master yarn-client --packages --num-executors 4 --driver-memory 512m
--executor-memory 512m --executor-cores 1 --keytab /xxx/user.headless.keytab
--principal x...@xx.com spark-*jar hiveTableInClient 180 {code}
After 8 hours, application fails with below error.
{code}
17/06/28 06:33:43 INFO ClientCnxn: Opening socket connection to server
xxx/xxx:2181. Will not attempt to authenticate using SASL (unknown error)
17/06/28 06:33:43 INFO ClientCnxn: Socket connection established to
xxx/xxx:2181, initiating session
17/06/28 06:33:43 INFO ClientCnxn: Session establishment complete on server
xxx/xxx:2181, sessionid = 0x25ced1d3ac20022, negotiated timeout = 9
17/06/28 06:33:43 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:33:45 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:33:48 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:33:52 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:34:02 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:34:12 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired{code}
Here, Jars pulled from "--packages" are not added into AM class path and that's
the reason why AM cannot get HBase tokens and failed after token expired.
was:
STR:
* Set below config in spark-default.conf
{code}
spark.yarn.security.credentials.hbase.enabled true
spark.hbase.connector.security.credentials.enabled false{code}
* Set below config in hdfs-site.xml
{code}
'dfs.namenode.delegation.token.max-lifetime':'4320'
'dfs.namenode.delegation.token.renew-interval':'2880' {code}
* Set below config in hbase-site.xml
{code}
'hbase.auth.token.max.lifetime': '2880' {code}
* Run an application with SHC package
spark-submit --class
org.apache.spark.sql.execution.datasources.hbase.examples.LRJobForDataSources
--master yarn-client --packages --num-executors 4 --driver-memory 512m
--executor-memory 512m --executor-cores 1 --keytab /xxx/user.headless.keytab
--principal x...@xx.com spark-*jar hiveTableInClient 180 {code}
After 8 hours, application fails with below error.
{code}
17/06/28 06:33:43 INFO ClientCnxn: Opening socket connection to server
xxx/xxx:2181. Will not attempt to authenticate using SASL (unknown error)
17/06/28 06:33:43 INFO ClientCnxn: Socket connection established to
xxx/xxx:2181, initiating session
17/06/28 06:33:43 INFO ClientCnxn: Session establishment complete on server
xxx/xxx:2181, sessionid = 0x25ced1d3ac20022, negotiated timeout = 9
17/06/28 06:33:43 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:33:45 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:33:48 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:33:52 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hado
[jira] [Updated] (SPARK-21377) Jars pulled from "--packages" are not added into AM classpath
[
https://issues.apache.org/jira/browse/SPARK-21377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-21377:
---
Description:
STR:
* Set below config in spark-default.conf
{code}
spark.yarn.security.credentials.hbase.enabled true
spark.hbase.connector.security.credentials.enabled false{code}
* Set below config in hdfs-site.xml
{code}
'dfs.namenode.delegation.token.max-lifetime':'4320'
'dfs.namenode.delegation.token.renew-interval':'2880' {code}
* Set below config in hbase-site.xml
{code}
'hbase.auth.token.max.lifetime': '2880' {code}
* Run an application with SHC package
spark-submit --class
org.apache.spark.sql.execution.datasources.hbase.examples.LRJobForDataSources
--master yarn-client --packages --num-executors 4 --driver-memory 512m
--executor-memory 512m --executor-cores 1 --keytab /xxx/user.headless.keytab
--principal x...@xx.com spark-*jar hiveTableInClient 180 {code}
After 8 hours, application fails with below error.
{code}
17/06/28 06:33:43 INFO ClientCnxn: Opening socket connection to server
xxx/xxx:2181. Will not attempt to authenticate using SASL (unknown error)
17/06/28 06:33:43 INFO ClientCnxn: Socket connection established to
xxx/xxx:2181, initiating session
17/06/28 06:33:43 INFO ClientCnxn: Session establishment complete on server
xxx/xxx:2181, sessionid = 0x25ced1d3ac20022, negotiated timeout = 9
17/06/28 06:33:43 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:33:45 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:33:48 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:33:52 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:34:02 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired
17/06/28 06:34:12 WARN AbstractRpcClient: Exception encountered while
connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Token has expired{code}
Here, Jars pulled from "--packages" are not added into AM class path and that's
the reason why AM cannot get HBase tokens and failed after token expired.
was:
STR:
* Set below config in spark-default.conf
{code}
spark.yarn.security.credentials.hbase.enabled true
spark.hbase.connector.security.credentials.enabled false{code}
* Set below config in hdfs-site.xml
{code}
'dfs.namenode.delegation.token.max-lifetime':'4320'
'dfs.namenode.delegation.token.renew-interval':'2880' {code}
* Run HDFSWordcount streaming app in yarn-cluster mode for 25 hrs.
After 25 hours, noticing that HDFS Wordcount job is hitting
HDFS_DELEGATION_TOKEN renewal issue.
{code}
17/06/28 10:49:47 WARN Client: Exception encountered while connecting to the
server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 230 for hrt_qa) is expired
17/06/28 10:49:47 WARN Client: Failed to cleanup staging dir
hdfs://mycluster0/user/hrt_qa/.sparkStaging/application_1498539861056_0015
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 230 for hrt_qa) is expired
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1554)
at org.apache.hadoop.ipc.Client.call(Client.java:1498){code}
> Jars pulled from "--packages" are not added into AM classpath
> -
>
> Key: SPARK-21377
> URL: https://issues.apache.org/jira/browse/SPARK-21377
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 2.1.0
>Reporter: Yesha Vora
>
> STR:
> * Set below config in spark-default.conf
> {code}
> spark.yarn.security.credentials.hbase.enabled true
> spark.hbase.connector.security.credentials.enabled false{code}
> * Set below config in hdfs-site.xml
> {code}
> 'dfs.namenode.delegation.token.max-lifetime':'4320'
> 'dfs.namenode.delegation.token.renew-interval':'2880' {code}
> * Set below config in hbase-site.xml
> {code}
> 'hbase.auth.token.max.lifetime': '2880' {code}
> * Run an appl
[jira] [Created] (SPARK-21377) Jars pulled from "--packages" are not added into AM classpath
Yesha Vora created SPARK-21377:
--
Summary: Jars pulled from "--packages" are not added into AM
classpath
Key: SPARK-21377
URL: https://issues.apache.org/jira/browse/SPARK-21377
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.1.0
Reporter: Yesha Vora
STR:
* Set below config in spark-default.conf
{code}
spark.yarn.security.credentials.hbase.enabled true
spark.hbase.connector.security.credentials.enabled false{code}
* Set below config in hdfs-site.xml
{code}
'dfs.namenode.delegation.token.max-lifetime':'4320'
'dfs.namenode.delegation.token.renew-interval':'2880' {code}
* Run HDFSWordcount streaming app in yarn-cluster mode for 25 hrs.
After 25 hours, noticing that HDFS Wordcount job is hitting
HDFS_DELEGATION_TOKEN renewal issue.
{code}
17/06/28 10:49:47 WARN Client: Exception encountered while connecting to the
server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 230 for hrt_qa) is expired
17/06/28 10:49:47 WARN Client: Failed to cleanup staging dir
hdfs://mycluster0/user/hrt_qa/.sparkStaging/application_1498539861056_0015
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 230 for hrt_qa) is expired
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1554)
at org.apache.hadoop.ipc.Client.call(Client.java:1498){code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-19158) ml.R example fails in yarn-cluster mode due to lacks of e1071 package
Yesha Vora created SPARK-19158:
--
Summary: ml.R example fails in yarn-cluster mode due to lacks of
e1071 package
Key: SPARK-19158
URL: https://issues.apache.org/jira/browse/SPARK-19158
Project: Spark
Issue Type: Bug
Components: Examples
Reporter: Yesha Vora
ml.R application fails in spark2 with yarn-cluster mode.
{code}
spark-submit --master yarn-cluster examples/src/main/r/ml/ml.R {code}
{code:title=application log}
17/01/03 04:35:30 INFO MemoryStore: Block broadcast_88 stored as values in
memory (estimated size 6.8 KB, free 407.6 MB)
17/01/03 04:35:30 INFO BufferedStreamThread: Error : requireNamespace("e1071",
quietly = TRUE) is not TRUE
17/01/03 04:35:30 ERROR Executor: Exception in task 0.0 in stage 65.0 (TID 65)
org.apache.spark.SparkException: R computation failed with
Error : requireNamespace("e1071", quietly = TRUE) is not TRUE
at org.apache.spark.api.r.RRunner.compute(RRunner.scala:108)
at org.apache.spark.api.r.BaseRRDD.compute(RRDD.scala:50)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/01/03 04:35:30 INFO CoarseGrainedExecutorBackend: Got assigned task 68
17/01/03 04:35:30 INFO Executor: Running task 3.0 in stage 65.0 (TID 68)
17/01/03 04:35:30 INFO BufferedStreamThread: Error : requireNamespace("e1071",
quietly = TRUE) is not TRUE
17/01/03 04:35:30 ERROR Executor: Exception in task 3.0 in stage 65.0 (TID 68)
org.apache.spark.SparkException: R computation failed with
Error : requireNamespace("e1071", quietly = TRUE) is not TRUE
Error : requireNamespace("e1071", quietly = TRUE) is not TRUE
at org.apache.spark.api.r.RRunner.compute(RRunner.scala:108)
at org.apache.spark.api.r.BaseRRDD.compute(RRDD.scala:50)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
17/01/03 04:35:30 INFO CoarseGrainedExecutorBackend: Got assigned task 70
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-19097) virtualenv example failed with conda due to ImportError: No module named ruamel.yaml.comments
Yesha Vora created SPARK-19097:
--
Summary: virtualenv example failed with conda due to ImportError:
No module named ruamel.yaml.comments
Key: SPARK-19097
URL: https://issues.apache.org/jira/browse/SPARK-19097
Project: Spark
Issue Type: Sub-task
Components: PySpark
Reporter: Yesha Vora
Spark version : 2
Steps:
* install conda on all nodes (python2.7) ( pip install conda )
* create requirement1.txt with "numpy > requirement1.txt "
* Run kmeans.py application in yarn-client mode.
{code}
spark-submit --master yarn --deploy-mode client --conf
"spark.pyspark.virtualenv.enabled=true" --conf
"spark.pyspark.virtualenv.type=conda" --conf
"spark.pyspark.virtualenv.requirements=/tmp/requirements1.txt" --conf
"spark.pyspark.virtualenv.bin.path=/usr/bin/conda" --jars
/usr/hadoop-client/lib/hadoop-lzo.jar kmeans.py /tmp/in/kmeans_data.txt 3{code}
{code:title=app log}
17/01/06 01:39:25 DEBUG PythonWorkerFactory: user.home=/home/yarn
17/01/06 01:39:25 DEBUG PythonWorkerFactory: Running command:/usr/bin/conda
create --prefix
/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0017/container_1483592608863_0017_01_03/virtualenv_application_1483592608863_0017_0
--file requirements1.txt -y
Traceback (most recent call last):
File "/usr/bin/conda", line 11, in
load_entry_point('conda==4.2.7', 'console_scripts', 'conda')()
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 561,
in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2631,
in load_entry_point
return ep.load()
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2291,
in load
return self.resolve()
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2297,
in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/usr/lib/python2.7/site-packages/conda/cli/__init__.py", line 8, in
from .main import main # NOQA
File "/usr/lib/python2.7/site-packages/conda/cli/main.py", line 46, in
from ..base.context import context
File "/usr/lib/python2.7/site-packages/conda/base/context.py", line 18, in
from ..common.configuration import (Configuration, MapParameter,
PrimitiveParameter,
File "/usr/lib/python2.7/site-packages/conda/common/configuration.py", line
40, in
from ruamel.yaml.comments import CommentedSeq, CommentedMap # pragma: no
cover
ImportError: No module named ruamel.yaml.comments
17/01/06 01:39:26 WARN BlockManager: Putting block rdd_3_0 failed due to an
exception
17/01/06 01:39:26 WARN BlockManager: Block rdd_3_0 could not be removed as it
was not found on disk or in memory
17/01/06 01:39:26 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
java.lang.RuntimeException: Fail to run command: /usr/bin/conda create --prefix
/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0017/container_1483592608863_0017_01_03/virtualenv_application_1483592608863_0017_0
--file requirements1.txt -y
at
org.apache.spark.api.python.PythonWorkerFactory.execCommand(PythonWorkerFactory.scala:142)
at
org.apache.spark.api.python.PythonWorkerFactory.setupVirtualEnv(PythonWorkerFactory.scala:124)
at
org.apache.spark.api.python.PythonWorkerFactory.(PythonWorkerFactory.scala:70)
at
org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
at
org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
at
scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:194)
at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:80)
at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:116)
at org.apache.spark.api.python.PythonRunner.compute(PythonRDD.scala:128)
at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:63)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:336)
at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:334)
at
org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:957)
at
org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:948)
at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:888)
at
org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:948)
at
org.apache.spark.storage.BlockManager.getOrElseUpdate(Blo
[jira] [Updated] (SPARK-19096) Kmeans.py application fails with virtualenv and due to parse error
[
https://issues.apache.org/jira/browse/SPARK-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-19096:
---
Issue Type: Sub-task (was: Bug)
Parent: SPARK-13587
> Kmeans.py application fails with virtualenv and due to parse error
>
>
> Key: SPARK-19096
> URL: https://issues.apache.org/jira/browse/SPARK-19096
> Project: Spark
> Issue Type: Sub-task
> Components: PySpark
>Reporter: Yesha Vora
>
> Spark version : 2
> Steps:
> * Install virtualenv ( pip install virtualenv)
> * create requirements.txt (pip freeze > /tmp/requirements.txt)
> * start kmeans.py application in yarn-client mode.
> The application fails with Runtime Exception
> {code:title=app log}
> 17/01/05 19:49:59 INFO deprecation: mapred.task.partition is deprecated.
> Instead, use mapreduce.task.partition
> 17/01/05 19:49:59 INFO deprecation: mapred.job.id is deprecated. Instead, use
> mapreduce.job.id
> Invalid requirement: 'pip freeze'
> Traceback (most recent call last):
> File
> "/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0006/container_1483592608863_0006_01_02/virtualenv_application_1483592608863_0006_0/lib/python2.7/site-packages/pip/req/req_install.py",
> line 82, in __init__
> req = Requirement(req)
> File
> "/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0006/container_1483592608863_0006_01_02/virtualenv_application_1483592608863_0006_0/lib/python2.7/site-packages/pip/_vendor/packaging/requirements.py",
> line 96, in __init__
> requirement_string[e.loc:e.loc + 8]))
> InvalidRequirement: Invalid requirement, parse error at "u'freeze'"
> 17/01/05 19:50:03 WARN BlockManager: Putting block rdd_3_0 failed due to an
> exception
> 17/01/05 19:50:03 WARN BlockManager: Block rdd_3_0 could not be removed as it
> was not found on disk or in memory
> 17/01/05 19:50:03 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
> {code}
> {code:title=job client log}
> 17/01/05 19:50:07 WARN TaskSetManager: Lost task 0.1 in stage 0.0 (TID 2,
> xxx.site, executor 1): java.lang.RuntimeException: Fail to run command:
> virtualenv_application_1483592608863_0006_1/bin/python -m pip --cache-dir
> /home/yarn install -r requirements.txt
> at
> org.apache.spark.api.python.PythonWorkerFactory.execCommand(PythonWorkerFactory.scala:142)
> at
> org.apache.spark.api.python.PythonWorkerFactory.setupVirtualEnv(PythonWorkerFactory.scala:128)
> at
> org.apache.spark.api.python.PythonWorkerFactory.(PythonWorkerFactory.scala:70)
> at
> org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
> at
> org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
> at
> scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:194)
> at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:80)
> at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:116)
> at org.apache.spark.api.python.PythonRunner.compute(PythonRDD.scala:128)
> at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:63)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:336)
> at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:334)
> at
> org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:957)
> at
> org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:948)
> at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:888)
> at
> org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:948)
> at
> org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:694)
> at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:334)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:285)
> at
> org.apache.spark.rdd.ZippedPartitionsRDD2.compute(ZippedPartitionsRDD.scala:89)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.a
[jira] [Commented] (SPARK-19096) Kmeans.py application fails with virtualenv and due to parse error
[
https://issues.apache.org/jira/browse/SPARK-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803223#comment-15803223
]
Yesha Vora commented on SPARK-19096:
This is valid bug. Thus reopening it and linking it with SPARK-13587
> Kmeans.py application fails with virtualenv and due to parse error
>
>
> Key: SPARK-19096
> URL: https://issues.apache.org/jira/browse/SPARK-19096
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Reporter: Yesha Vora
>
> Spark version : 2
> Steps:
> * Install virtualenv ( pip install virtualenv)
> * create requirements.txt (pip freeze > /tmp/requirements.txt)
> * start kmeans.py application in yarn-client mode.
> The application fails with Runtime Exception
> {code:title=app log}
> 17/01/05 19:49:59 INFO deprecation: mapred.task.partition is deprecated.
> Instead, use mapreduce.task.partition
> 17/01/05 19:49:59 INFO deprecation: mapred.job.id is deprecated. Instead, use
> mapreduce.job.id
> Invalid requirement: 'pip freeze'
> Traceback (most recent call last):
> File
> "/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0006/container_1483592608863_0006_01_02/virtualenv_application_1483592608863_0006_0/lib/python2.7/site-packages/pip/req/req_install.py",
> line 82, in __init__
> req = Requirement(req)
> File
> "/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0006/container_1483592608863_0006_01_02/virtualenv_application_1483592608863_0006_0/lib/python2.7/site-packages/pip/_vendor/packaging/requirements.py",
> line 96, in __init__
> requirement_string[e.loc:e.loc + 8]))
> InvalidRequirement: Invalid requirement, parse error at "u'freeze'"
> 17/01/05 19:50:03 WARN BlockManager: Putting block rdd_3_0 failed due to an
> exception
> 17/01/05 19:50:03 WARN BlockManager: Block rdd_3_0 could not be removed as it
> was not found on disk or in memory
> 17/01/05 19:50:03 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
> {code}
> {code:title=job client log}
> 17/01/05 19:50:07 WARN TaskSetManager: Lost task 0.1 in stage 0.0 (TID 2,
> xxx.site, executor 1): java.lang.RuntimeException: Fail to run command:
> virtualenv_application_1483592608863_0006_1/bin/python -m pip --cache-dir
> /home/yarn install -r requirements.txt
> at
> org.apache.spark.api.python.PythonWorkerFactory.execCommand(PythonWorkerFactory.scala:142)
> at
> org.apache.spark.api.python.PythonWorkerFactory.setupVirtualEnv(PythonWorkerFactory.scala:128)
> at
> org.apache.spark.api.python.PythonWorkerFactory.(PythonWorkerFactory.scala:70)
> at
> org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
> at
> org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
> at
> scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:194)
> at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:80)
> at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:116)
> at org.apache.spark.api.python.PythonRunner.compute(PythonRDD.scala:128)
> at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:63)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:336)
> at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:334)
> at
> org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:957)
> at
> org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:948)
> at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:888)
> at
> org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:948)
> at
> org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:694)
> at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:334)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:285)
> at
> org.apache.spark.rdd.ZippedPartitionsRDD2.compute(ZippedPartitionsRDD.scala:89)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> at org.apache
[jira] [Reopened] (SPARK-19096) Kmeans.py application fails with virtualenv and due to parse error
[
https://issues.apache.org/jira/browse/SPARK-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora reopened SPARK-19096:
> Kmeans.py application fails with virtualenv and due to parse error
>
>
> Key: SPARK-19096
> URL: https://issues.apache.org/jira/browse/SPARK-19096
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Reporter: Yesha Vora
>
> Spark version : 2
> Steps:
> * Install virtualenv ( pip install virtualenv)
> * create requirements.txt (pip freeze > /tmp/requirements.txt)
> * start kmeans.py application in yarn-client mode.
> The application fails with Runtime Exception
> {code:title=app log}
> 17/01/05 19:49:59 INFO deprecation: mapred.task.partition is deprecated.
> Instead, use mapreduce.task.partition
> 17/01/05 19:49:59 INFO deprecation: mapred.job.id is deprecated. Instead, use
> mapreduce.job.id
> Invalid requirement: 'pip freeze'
> Traceback (most recent call last):
> File
> "/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0006/container_1483592608863_0006_01_02/virtualenv_application_1483592608863_0006_0/lib/python2.7/site-packages/pip/req/req_install.py",
> line 82, in __init__
> req = Requirement(req)
> File
> "/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0006/container_1483592608863_0006_01_02/virtualenv_application_1483592608863_0006_0/lib/python2.7/site-packages/pip/_vendor/packaging/requirements.py",
> line 96, in __init__
> requirement_string[e.loc:e.loc + 8]))
> InvalidRequirement: Invalid requirement, parse error at "u'freeze'"
> 17/01/05 19:50:03 WARN BlockManager: Putting block rdd_3_0 failed due to an
> exception
> 17/01/05 19:50:03 WARN BlockManager: Block rdd_3_0 could not be removed as it
> was not found on disk or in memory
> 17/01/05 19:50:03 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
> {code}
> {code:title=job client log}
> 17/01/05 19:50:07 WARN TaskSetManager: Lost task 0.1 in stage 0.0 (TID 2,
> xxx.site, executor 1): java.lang.RuntimeException: Fail to run command:
> virtualenv_application_1483592608863_0006_1/bin/python -m pip --cache-dir
> /home/yarn install -r requirements.txt
> at
> org.apache.spark.api.python.PythonWorkerFactory.execCommand(PythonWorkerFactory.scala:142)
> at
> org.apache.spark.api.python.PythonWorkerFactory.setupVirtualEnv(PythonWorkerFactory.scala:128)
> at
> org.apache.spark.api.python.PythonWorkerFactory.(PythonWorkerFactory.scala:70)
> at
> org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
> at
> org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
> at
> scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:194)
> at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:80)
> at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:116)
> at org.apache.spark.api.python.PythonRunner.compute(PythonRDD.scala:128)
> at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:63)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:336)
> at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:334)
> at
> org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:957)
> at
> org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:948)
> at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:888)
> at
> org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:948)
> at
> org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:694)
> at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:334)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:285)
> at
> org.apache.spark.rdd.ZippedPartitionsRDD2.compute(ZippedPartitionsRDD.scala:89)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at
> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
>
[jira] [Updated] (SPARK-19095) virtualenv example does not work in yarn cluster mode
[
https://issues.apache.org/jira/browse/SPARK-19095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-19095:
---
Issue Type: Sub-task (was: Bug)
Parent: SPARK-13587
> virtualenv example does not work in yarn cluster mode
> -
>
> Key: SPARK-19095
> URL: https://issues.apache.org/jira/browse/SPARK-19095
> Project: Spark
> Issue Type: Sub-task
>Reporter: Yesha Vora
>Priority: Critical
>
> Spark version: 2
> Steps:
> * install virtualenv on all nodes
> * create requirement1.txt with "numpy > requirement1.txt "
> * Run kmeans.py application in yarn-cluster mode.
> {code}
> spark-submit --master yarn --deploy-mode cluster --conf
> "spark.pyspark.virtualenv.enabled=true" --conf
> "spark.pyspark.virtualenv.type=native" --conf
> "spark.pyspark.virtualenv.requirements=/tmp/requirements1.txt" --conf
> "spark.pyspark.virtualenv.bin.path=/usr/bin/virtualenv" --jars
> /usr/hdp/current/hadoop-client/lib/hadoop-lzo.jar kmeans.py
> /tmp/in/kmeans_data.txt 3{code}
> The application fails to find numpy.
> {code}
> LogType:stdout
> Log Upload Time:Thu Jan 05 20:05:49 + 2017
> LogLength:134
> Log Contents:
> Traceback (most recent call last):
> File "kmeans.py", line 27, in
> import numpy as np
> ImportError: No module named numpy
> End of LogType:stdout
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Reopened] (SPARK-19095) virtualenv example does not work in yarn cluster mode
[
https://issues.apache.org/jira/browse/SPARK-19095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora reopened SPARK-19095:
This is valid bug. Thus reopening it and linking it with SPARK-13587.
> virtualenv example does not work in yarn cluster mode
> -
>
> Key: SPARK-19095
> URL: https://issues.apache.org/jira/browse/SPARK-19095
> Project: Spark
> Issue Type: Bug
>Reporter: Yesha Vora
>Priority: Critical
>
> Spark version: 2
> Steps:
> * install virtualenv on all nodes
> * create requirement1.txt with "numpy > requirement1.txt "
> * Run kmeans.py application in yarn-cluster mode.
> {code}
> spark-submit --master yarn --deploy-mode cluster --conf
> "spark.pyspark.virtualenv.enabled=true" --conf
> "spark.pyspark.virtualenv.type=native" --conf
> "spark.pyspark.virtualenv.requirements=/tmp/requirements1.txt" --conf
> "spark.pyspark.virtualenv.bin.path=/usr/bin/virtualenv" --jars
> /usr/hdp/current/hadoop-client/lib/hadoop-lzo.jar kmeans.py
> /tmp/in/kmeans_data.txt 3{code}
> The application fails to find numpy.
> {code}
> LogType:stdout
> Log Upload Time:Thu Jan 05 20:05:49 + 2017
> LogLength:134
> Log Contents:
> Traceback (most recent call last):
> File "kmeans.py", line 27, in
> import numpy as np
> ImportError: No module named numpy
> End of LogType:stdout
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-19096) Kmeans.py application fails with virtualenv and due to parse error
[
https://issues.apache.org/jira/browse/SPARK-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-19096:
---
Description:
Spark version : 2
Steps:
* Install virtualenv ( pip install virtualenv)
* create requirements.txt (pip freeze > /tmp/requirements.txt)
* start kmeans.py application in yarn-client mode.
The application fails with Runtime Exception
{code:title=app log}
17/01/05 19:49:59 INFO deprecation: mapred.task.partition is deprecated.
Instead, use mapreduce.task.partition
17/01/05 19:49:59 INFO deprecation: mapred.job.id is deprecated. Instead, use
mapreduce.job.id
Invalid requirement: 'pip freeze'
Traceback (most recent call last):
File
"/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0006/container_1483592608863_0006_01_02/virtualenv_application_1483592608863_0006_0/lib/python2.7/site-packages/pip/req/req_install.py",
line 82, in __init__
req = Requirement(req)
File
"/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0006/container_1483592608863_0006_01_02/virtualenv_application_1483592608863_0006_0/lib/python2.7/site-packages/pip/_vendor/packaging/requirements.py",
line 96, in __init__
requirement_string[e.loc:e.loc + 8]))
InvalidRequirement: Invalid requirement, parse error at "u'freeze'"
17/01/05 19:50:03 WARN BlockManager: Putting block rdd_3_0 failed due to an
exception
17/01/05 19:50:03 WARN BlockManager: Block rdd_3_0 could not be removed as it
was not found on disk or in memory
17/01/05 19:50:03 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
{code}
{code:title=job client log}
17/01/05 19:50:07 WARN TaskSetManager: Lost task 0.1 in stage 0.0 (TID 2,
xxx.site, executor 1): java.lang.RuntimeException: Fail to run command:
virtualenv_application_1483592608863_0006_1/bin/python -m pip --cache-dir
/home/yarn install -r requirements.txt
at
org.apache.spark.api.python.PythonWorkerFactory.execCommand(PythonWorkerFactory.scala:142)
at
org.apache.spark.api.python.PythonWorkerFactory.setupVirtualEnv(PythonWorkerFactory.scala:128)
at
org.apache.spark.api.python.PythonWorkerFactory.(PythonWorkerFactory.scala:70)
at
org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
at
org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
at
scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:194)
at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:80)
at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:116)
at org.apache.spark.api.python.PythonRunner.compute(PythonRDD.scala:128)
at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:63)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:336)
at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:334)
at
org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:957)
at
org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:948)
at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:888)
at
org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:948)
at
org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:694)
at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:334)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:285)
at
org.apache.spark.rdd.ZippedPartitionsRDD2.compute(ZippedPartitionsRDD.scala:89)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745){code}
was:
Steps:
* Install virtualenv ( pip install virtualenv)
* create requirements.txt (pip freeze > /tmp/requirements.txt)
* start kmeans.py application in yarn-client mode.
The application fails with Runtime
[jira] [Updated] (SPARK-19095) virtualenv example does not work in yarn cluster mode
[
https://issues.apache.org/jira/browse/SPARK-19095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-19095:
---
Description:
Spark version: 2
Steps:
* install virtualenv on all nodes
* create requirement1.txt with "numpy > requirement1.txt "
* Run kmeans.py application in yarn-cluster mode.
{code}
spark-submit --master yarn --deploy-mode cluster --conf
"spark.pyspark.virtualenv.enabled=true" --conf
"spark.pyspark.virtualenv.type=native" --conf
"spark.pyspark.virtualenv.requirements=/tmp/requirements1.txt" --conf
"spark.pyspark.virtualenv.bin.path=/usr/bin/virtualenv" --jars
/usr/hdp/current/hadoop-client/lib/hadoop-lzo.jar kmeans.py
/tmp/in/kmeans_data.txt 3{code}
The application fails to find numpy.
{code}
LogType:stdout
Log Upload Time:Thu Jan 05 20:05:49 + 2017
LogLength:134
Log Contents:
Traceback (most recent call last):
File "kmeans.py", line 27, in
import numpy as np
ImportError: No module named numpy
End of LogType:stdout
{code}
was:
Steps:
* install virtualenv on all nodes
* create requirement1.txt with "numpy > requirement1.txt "
* Run kmeans.py application in yarn-cluster mode.
{code}
spark-submit --master yarn --deploy-mode cluster --conf
"spark.pyspark.virtualenv.enabled=true" --conf
"spark.pyspark.virtualenv.type=native" --conf
"spark.pyspark.virtualenv.requirements=/tmp/requirements1.txt" --conf
"spark.pyspark.virtualenv.bin.path=/usr/bin/virtualenv" --jars
/usr/hdp/current/hadoop-client/lib/hadoop-lzo.jar kmeans.py
/tmp/in/kmeans_data.txt 3{code}
The application fails to find numpy.
{code}
LogType:stdout
Log Upload Time:Thu Jan 05 20:05:49 + 2017
LogLength:134
Log Contents:
Traceback (most recent call last):
File "kmeans.py", line 27, in
import numpy as np
ImportError: No module named numpy
End of LogType:stdout
{code}
> virtualenv example does not work in yarn cluster mode
> -
>
> Key: SPARK-19095
> URL: https://issues.apache.org/jira/browse/SPARK-19095
> Project: Spark
> Issue Type: Bug
>Reporter: Yesha Vora
>Priority: Critical
>
> Spark version: 2
> Steps:
> * install virtualenv on all nodes
> * create requirement1.txt with "numpy > requirement1.txt "
> * Run kmeans.py application in yarn-cluster mode.
> {code}
> spark-submit --master yarn --deploy-mode cluster --conf
> "spark.pyspark.virtualenv.enabled=true" --conf
> "spark.pyspark.virtualenv.type=native" --conf
> "spark.pyspark.virtualenv.requirements=/tmp/requirements1.txt" --conf
> "spark.pyspark.virtualenv.bin.path=/usr/bin/virtualenv" --jars
> /usr/hdp/current/hadoop-client/lib/hadoop-lzo.jar kmeans.py
> /tmp/in/kmeans_data.txt 3{code}
> The application fails to find numpy.
> {code}
> LogType:stdout
> Log Upload Time:Thu Jan 05 20:05:49 + 2017
> LogLength:134
> Log Contents:
> Traceback (most recent call last):
> File "kmeans.py", line 27, in
> import numpy as np
> ImportError: No module named numpy
> End of LogType:stdout
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-19096) Kmeans.py application fails with virtualenv and due to parse error
Yesha Vora created SPARK-19096:
--
Summary: Kmeans.py application fails with virtualenv and due to
parse error
Key: SPARK-19096
URL: https://issues.apache.org/jira/browse/SPARK-19096
Project: Spark
Issue Type: Bug
Components: PySpark
Reporter: Yesha Vora
Steps:
* Install virtualenv ( pip install virtualenv)
* create requirements.txt (pip freeze > /tmp/requirements.txt)
* start kmeans.py application in yarn-client mode.
The application fails with Runtime Exception
{code:title=app log}
17/01/05 19:49:59 INFO deprecation: mapred.task.partition is deprecated.
Instead, use mapreduce.task.partition
17/01/05 19:49:59 INFO deprecation: mapred.job.id is deprecated. Instead, use
mapreduce.job.id
Invalid requirement: 'pip freeze'
Traceback (most recent call last):
File
"/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0006/container_1483592608863_0006_01_02/virtualenv_application_1483592608863_0006_0/lib/python2.7/site-packages/pip/req/req_install.py",
line 82, in __init__
req = Requirement(req)
File
"/grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1483592608863_0006/container_1483592608863_0006_01_02/virtualenv_application_1483592608863_0006_0/lib/python2.7/site-packages/pip/_vendor/packaging/requirements.py",
line 96, in __init__
requirement_string[e.loc:e.loc + 8]))
InvalidRequirement: Invalid requirement, parse error at "u'freeze'"
17/01/05 19:50:03 WARN BlockManager: Putting block rdd_3_0 failed due to an
exception
17/01/05 19:50:03 WARN BlockManager: Block rdd_3_0 could not be removed as it
was not found on disk or in memory
17/01/05 19:50:03 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
{code}
{code:title=job client log}
17/01/05 19:50:07 WARN TaskSetManager: Lost task 0.1 in stage 0.0 (TID 2,
xxx.site, executor 1): java.lang.RuntimeException: Fail to run command:
virtualenv_application_1483592608863_0006_1/bin/python -m pip --cache-dir
/home/yarn install -r requirements.txt
at
org.apache.spark.api.python.PythonWorkerFactory.execCommand(PythonWorkerFactory.scala:142)
at
org.apache.spark.api.python.PythonWorkerFactory.setupVirtualEnv(PythonWorkerFactory.scala:128)
at
org.apache.spark.api.python.PythonWorkerFactory.(PythonWorkerFactory.scala:70)
at
org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
at
org.apache.spark.SparkEnv$$anonfun$createPythonWorker$1.apply(SparkEnv.scala:117)
at
scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:194)
at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:80)
at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:116)
at org.apache.spark.api.python.PythonRunner.compute(PythonRDD.scala:128)
at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:63)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:336)
at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:334)
at
org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:957)
at
org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:948)
at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:888)
at
org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:948)
at
org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:694)
at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:334)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:285)
at
org.apache.spark.rdd.ZippedPartitionsRDD2.compute(ZippedPartitionsRDD.scala:89)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#63
[jira] [Created] (SPARK-19095) virtualenv example does not work in yarn cluster mode
Yesha Vora created SPARK-19095:
--
Summary: virtualenv example does not work in yarn cluster mode
Key: SPARK-19095
URL: https://issues.apache.org/jira/browse/SPARK-19095
Project: Spark
Issue Type: Bug
Reporter: Yesha Vora
Priority: Critical
Steps:
* install virtualenv on all nodes
* create requirement1.txt with "numpy > requirement1.txt "
* Run kmeans.py application in yarn-cluster mode.
{code}
spark-submit --master yarn --deploy-mode cluster --conf
"spark.pyspark.virtualenv.enabled=true" --conf
"spark.pyspark.virtualenv.type=native" --conf
"spark.pyspark.virtualenv.requirements=/tmp/requirements1.txt" --conf
"spark.pyspark.virtualenv.bin.path=/usr/bin/virtualenv" --jars
/usr/hdp/current/hadoop-client/lib/hadoop-lzo.jar kmeans.py
/tmp/in/kmeans_data.txt 3{code}
The application fails to find numpy.
{code}
LogType:stdout
Log Upload Time:Thu Jan 05 20:05:49 + 2017
LogLength:134
Log Contents:
Traceback (most recent call last):
File "kmeans.py", line 27, in
import numpy as np
ImportError: No module named numpy
End of LogType:stdout
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-17111) Spark2 History server got "Failed to load application attempt"
[
https://issues.apache.org/jira/browse/SPARK-17111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-17111:
---
Description:
Steps followed:
* Run sparkPi application in yarn-client mode
* SparkPi app (application_1471416622386_0083) finished at 09:03:37. At
09:03:49, Spark2 history server rest api was used to get app data. But it
failed with "no such app: application_1471416622386_0083".
{code}
http://xxx:18081/api/v1/applications/application_1471416622386_0083/executors
HTTP Code: 404
HTTP Data: no such app: application_1471416622386_0083{code}
{code:title=spark history server log}
16/08/17 09:00:10 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0083.inprogress
16/08/17 09:03:41 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0083
16/08/17 09:03:49 INFO SecurityManager: Changing view acls to: spark
16/08/17 09:03:49 INFO SecurityManager: Changing modify acls to: spark
16/08/17 09:03:49 INFO SecurityManager: Changing view acls groups to:
16/08/17 09:03:49 INFO SecurityManager: Changing modify acls groups to:
16/08/17 09:03:49 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(spark); groups
with view permissions: Set(); users with modify permissions: Set(spark);
groups with modify permissions: Set()
16/08/17 09:03:49 INFO ApplicationCache: Failed to load application attempt
application_1471416622386_0083/None
16/08/17 09:14:31 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0066_1.inprogress
16/08/17 09:32:54 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0073_1.inprogress
16/08/17 09:36:25 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0074_1.inprogress
16/08/17 09:43:06 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0076_1.inprogress
16/08/17 17:53:19 INFO SecurityManager: Changing view acls to: spark
16/08/17 17:53:19 INFO SecurityManager: Changing modify acls to: spark
16/08/17 17:53:19 INFO SecurityManager: Changing view acls groups to:
16/08/17 17:53:19 INFO SecurityManager: Changing modify acls groups to:
16/08/17 17:53:19 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(spark); groups
with view permissions: Set(); users with modify permissions: Set(spark);
groups with modify permissions: Set()
16/08/17 17:53:19 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0083
16/08/17 17:53:31 INFO SecurityManager: Changing acls enabled to: false
16/08/17 17:53:31 INFO SecurityManager: Changing admin acls to:
16/08/17 17:53:31 INFO SecurityManager: Changing view acls to: hrt_qa
16/08/17 17:53:31 INFO SecurityManager: Changing admin acls groups to:
16/08/17 17:53:31 INFO SecurityManager: Changing view acls groups to:
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@34cb487a{/history/application_1471416622386_0083/jobs,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@9927642{/history/application_1471416622386_0083/jobs/json,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@74151089{/history/application_1471416622386_0083/jobs/job,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@66605367{/history/application_1471416622386_0083/jobs/job/json,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@38c54bf5{/history/application_1471416622386_0083/stages,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@707a18ef{/history/application_1471416622386_0083/stages/json,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@42afbe71{/history/application_1471416622386_0083/stages/stage,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@5a3ca0e2{/history/application_1471416622386_0083/stages/stage/json,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@b7d5e57{/history/application_1471416622386_0083/stages/pool,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@4538d13e{/history/application_1471416622386_0083/stages/pool/json,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@6119809f{/history/application_1471416622386_0083/storage,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@7a269b27{/history/application_1471416622386_0083/storag
[jira] [Created] (SPARK-17111) Spark2 History server got "Failed to load application attempt"
Yesha Vora created SPARK-17111:
--
Summary: Spark2 History server got "Failed to load application
attempt"
Key: SPARK-17111
URL: https://issues.apache.org/jira/browse/SPARK-17111
Project: Spark
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yesha Vora
Steps followed:
* Run sparkPi application in yarn-client mode
* SparkPi app (application_1471416622386_0083) finished at 09:03:37. At
09:03:49, Spark2 history server rest api was used to get app data. But it
failed with "no such app: application_1471416622386_0083".
{code}
http://nat-d7-zods-spark2-1.openstacklocal:18081/api/v1/applications/application_1471416622386_0083/executors
HTTP Code: 404
HTTP Data: no such app: application_1471416622386_0083{code}
{code:title=spark history server log}
16/08/17 09:00:10 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0083.inprogress
16/08/17 09:03:41 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0083
16/08/17 09:03:49 INFO SecurityManager: Changing view acls to: spark
16/08/17 09:03:49 INFO SecurityManager: Changing modify acls to: spark
16/08/17 09:03:49 INFO SecurityManager: Changing view acls groups to:
16/08/17 09:03:49 INFO SecurityManager: Changing modify acls groups to:
16/08/17 09:03:49 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(spark); groups
with view permissions: Set(); users with modify permissions: Set(spark);
groups with modify permissions: Set()
16/08/17 09:03:49 INFO ApplicationCache: Failed to load application attempt
application_1471416622386_0083/None
16/08/17 09:14:31 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0066_1.inprogress
16/08/17 09:32:54 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0073_1.inprogress
16/08/17 09:36:25 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0074_1.inprogress
16/08/17 09:43:06 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0076_1.inprogress
16/08/17 17:53:19 INFO SecurityManager: Changing view acls to: spark
16/08/17 17:53:19 INFO SecurityManager: Changing modify acls to: spark
16/08/17 17:53:19 INFO SecurityManager: Changing view acls groups to:
16/08/17 17:53:19 INFO SecurityManager: Changing modify acls groups to:
16/08/17 17:53:19 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(spark); groups
with view permissions: Set(); users with modify permissions: Set(spark);
groups with modify permissions: Set()
16/08/17 17:53:19 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/spark2-history/application_1471416622386_0083
16/08/17 17:53:31 INFO SecurityManager: Changing acls enabled to: false
16/08/17 17:53:31 INFO SecurityManager: Changing admin acls to:
16/08/17 17:53:31 INFO SecurityManager: Changing view acls to: hrt_qa
16/08/17 17:53:31 INFO SecurityManager: Changing admin acls groups to:
16/08/17 17:53:31 INFO SecurityManager: Changing view acls groups to:
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@34cb487a{/history/application_1471416622386_0083/jobs,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@9927642{/history/application_1471416622386_0083/jobs/json,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@74151089{/history/application_1471416622386_0083/jobs/job,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@66605367{/history/application_1471416622386_0083/jobs/job/json,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@38c54bf5{/history/application_1471416622386_0083/stages,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@707a18ef{/history/application_1471416622386_0083/stages/json,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@42afbe71{/history/application_1471416622386_0083/stages/stage,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@5a3ca0e2{/history/application_1471416622386_0083/stages/stage/json,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@b7d5e57{/history/application_1471416622386_0083/stages/pool,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@4538d13e{/history/application_1471416622386_0083/stages/pool/json,null,AVAILABLE}
16/08/17 17:53:31 INFO ContextHandler: Started
o.s.j.s.ServletContextHandler@6119809f
[jira] [Created] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled
Yesha Vora created SPARK-16988:
--
Summary: spark history server log needs to be fixed to show https
url when ssl is enabled
Key: SPARK-16988
URL: https://issues.apache.org/jira/browse/SPARK-16988
Project: Spark
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yesha Vora
When spark ssl is enabled, spark history server ui ( http://host:port) is
redirected to https://host:port+400.
So, spark history server log should be updated to print https url instead http
url
{code:title=spark HS log}
16/08/09 15:21:11 INFO ServerConnector: Started
ServerConnector@3970a5ee{SSL-HTTP/1.1}{0.0.0.0:18481}
16/08/09 15:21:11 INFO Server: Started @4023ms
16/08/09 15:21:11 INFO Utils: Successfully started service on port 18081.
16/08/09 15:21:11 INFO HistoryServer: Bound HistoryServer to 0.0.0.0, and
started at http://xxx:18081
16/08/09 15:22:52 INFO FsHistoryProvider: Replaying log path:
hdfs://xxx:8020/yy/application_1470756121646_0001.inprogress{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-16987) Add spark-default.conf property to define https port for spark history server
Yesha Vora created SPARK-16987: -- Summary: Add spark-default.conf property to define https port for spark history server Key: SPARK-16987 URL: https://issues.apache.org/jira/browse/SPARK-16987 Project: Spark Issue Type: Bug Affects Versions: 2.0.0 Reporter: Yesha Vora With SPARK-2750, Spark History server UI becomes accessible on https port. Currently, https port is pre-defined to http port + 400. Spark History server UI https port should not be pre-defined but it should be configurable. Thus, spark should to introduce new property to make spark history server https port configurable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-16655) Spark thrift server application is not stopped if its in ACCEPTED stage
Yesha Vora created SPARK-16655: -- Summary: Spark thrift server application is not stopped if its in ACCEPTED stage Key: SPARK-16655 URL: https://issues.apache.org/jira/browse/SPARK-16655 Project: Spark Issue Type: Bug Affects Versions: 2.0.0 Reporter: Yesha Vora When spark-thriftserver is started in yarn-client mode, It starts a yarn application. If yarn application is in ACCEPTED stage and stop operation is performed on spark thrift server, yarn application does not get killed/stopped. On stop operation, spark thriftserver should stop the yarn application. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16595) Spark History server Rest Api gives Application not found error for yarn-cluster mode
[
https://issues.apache.org/jira/browse/SPARK-16595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381760#comment-15381760
]
Yesha Vora commented on SPARK-16595:
[~sowen], SPARK-15923 is referring to yarn-client mode. I opened this jira
because spark HS rest api threw app not found error in yarn-cluster mode too.
Sorry for not explicitly mentioning earlier. Since this issue is different from
SPARK-15923. Thus, reopening this jira.
> Spark History server Rest Api gives Application not found error for
> yarn-cluster mode
> -
>
> Key: SPARK-16595
> URL: https://issues.apache.org/jira/browse/SPARK-16595
> Project: Spark
> Issue Type: Bug
>Affects Versions: 1.6.1
>Reporter: Yesha Vora
>
> Scenario:
> * Start SparkPi application in Spark1 using yarn-cluster mode
> (application_1468686376753_0041)
> * After application finishes validate application exists in respective Spark
> History server.
> {code}
> Error loading url
> http://xx.xx.xx.xx:18080/api/v1/applications/application_1468686376753_0041/1/executors
> HTTP Code: 404
> HTTP Data: no such app: application_1468686376753_0041{code}
> {code:title=spark HS log}
> 16/07/16 15:55:29 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049.inprogress
> 16/07/16 15:56:20 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049
> 16/07/16 16:23:14 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061.inprogress
> 16/07/16 16:24:14 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061
> 16/07/16 17:42:32 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553.inprogress
> 16/07/16 17:43:22 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553
> 16/07/16 17:43:44 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376.inprogress
> 16/07/16 17:44:34 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376
> 16/07/16 18:53:10 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0041_1.inprogress
> 16/07/16 19:03:26 INFO PackagesResourceConfig: Scanning for root resource and
> provider classes in the packages:
> org.apache.spark.status.api.v1
> 16/07/16 19:03:35 INFO ScanningResourceConfig: Root resource classes found:
> class org.apache.spark.status.api.v1.ApiRootResource
> 16/07/16 19:03:35 INFO ScanningResourceConfig: Provider classes found:
> class org.apache.spark.status.api.v1.JacksonMessageWriter
> 16/07/16 19:03:35 INFO WebApplicationImpl: Initiating Jersey application,
> version 'Jersey: 1.9 09/02/2011 11:17 AM'
> 16/07/16 19:03:36 INFO SecurityManager: Changing view acls to: spark
> 16/07/16 19:03:36 INFO SecurityManager: Changing modify acls to: spark
> 16/07/16 19:03:36 INFO SecurityManager: SecurityManager: authentication
> disabled; ui acls disabled; users with view permissions: Set(spark); users
> with modify permissions: Set(spark)
> 16/07/16 19:03:36 INFO ApplicationCache: Failed to load application attempt
> application_1468686376753_0041/Some(1)
> 16/07/16 19:04:21 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043.inprogress
> 16/07/16 19:12:02 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
> 16/07/16 19:16:11 INFO SecurityManager: Changing view acls to: spark
> 16/07/16 19:16:11 INFO SecurityManager: Changing modify acls to: spark
> 16/07/16 19:16:11 INFO SecurityManager: SecurityManager: authentication
> disabled; ui acls disabled; users with view permissions: Set(spark); users
> with modify permissions: Set(spark)
> 16/07/16 19:16:11 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
> 16/07/16 19:16:22 INFO SecurityManager: Changing acls enabled to: false
> 16/07/16 19:16:22 INFO SecurityManager: Changing admin acls to:
> 16/07/16 19:16:22 INFO SecurityManager: Changing view acls to: hrt_qa{code}
> {code}
> hdfs@xxx:/var/log/spark$ hdfs dfs -ls /spark-history/
> Found 8 items
> -rwxrwx--- 3 hrt_qa hadoop 28793 2016-07-16 15:56
> /spark-history/application_1468678823755_0049
> -rwxrwx--- 3 hrt_qa hadoop 28763 2016-07-16 16:24
> /spark-history/application_1468678823755_0061
> -rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59
> /spark-history/application_1468686376753_0041_1
> -r
[jira] [Reopened] (SPARK-16595) Spark History server Rest Api gives Application not found error for yarn-cluster mode
[
https://issues.apache.org/jira/browse/SPARK-16595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora reopened SPARK-16595:
> Spark History server Rest Api gives Application not found error for
> yarn-cluster mode
> -
>
> Key: SPARK-16595
> URL: https://issues.apache.org/jira/browse/SPARK-16595
> Project: Spark
> Issue Type: Bug
>Affects Versions: 1.6.1
>Reporter: Yesha Vora
>
> Scenario:
> * Start SparkPi application in Spark1 using yarn-cluster mode
> (application_1468686376753_0041)
> * After application finishes validate application exists in respective Spark
> History server.
> {code}
> Error loading url
> http://xx.xx.xx.xx:18080/api/v1/applications/application_1468686376753_0041/1/executors
> HTTP Code: 404
> HTTP Data: no such app: application_1468686376753_0041{code}
> {code:title=spark HS log}
> 16/07/16 15:55:29 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049.inprogress
> 16/07/16 15:56:20 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049
> 16/07/16 16:23:14 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061.inprogress
> 16/07/16 16:24:14 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061
> 16/07/16 17:42:32 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553.inprogress
> 16/07/16 17:43:22 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553
> 16/07/16 17:43:44 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376.inprogress
> 16/07/16 17:44:34 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376
> 16/07/16 18:53:10 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0041_1.inprogress
> 16/07/16 19:03:26 INFO PackagesResourceConfig: Scanning for root resource and
> provider classes in the packages:
> org.apache.spark.status.api.v1
> 16/07/16 19:03:35 INFO ScanningResourceConfig: Root resource classes found:
> class org.apache.spark.status.api.v1.ApiRootResource
> 16/07/16 19:03:35 INFO ScanningResourceConfig: Provider classes found:
> class org.apache.spark.status.api.v1.JacksonMessageWriter
> 16/07/16 19:03:35 INFO WebApplicationImpl: Initiating Jersey application,
> version 'Jersey: 1.9 09/02/2011 11:17 AM'
> 16/07/16 19:03:36 INFO SecurityManager: Changing view acls to: spark
> 16/07/16 19:03:36 INFO SecurityManager: Changing modify acls to: spark
> 16/07/16 19:03:36 INFO SecurityManager: SecurityManager: authentication
> disabled; ui acls disabled; users with view permissions: Set(spark); users
> with modify permissions: Set(spark)
> 16/07/16 19:03:36 INFO ApplicationCache: Failed to load application attempt
> application_1468686376753_0041/Some(1)
> 16/07/16 19:04:21 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043.inprogress
> 16/07/16 19:12:02 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
> 16/07/16 19:16:11 INFO SecurityManager: Changing view acls to: spark
> 16/07/16 19:16:11 INFO SecurityManager: Changing modify acls to: spark
> 16/07/16 19:16:11 INFO SecurityManager: SecurityManager: authentication
> disabled; ui acls disabled; users with view permissions: Set(spark); users
> with modify permissions: Set(spark)
> 16/07/16 19:16:11 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
> 16/07/16 19:16:22 INFO SecurityManager: Changing acls enabled to: false
> 16/07/16 19:16:22 INFO SecurityManager: Changing admin acls to:
> 16/07/16 19:16:22 INFO SecurityManager: Changing view acls to: hrt_qa{code}
> {code}
> hdfs@xxx:/var/log/spark$ hdfs dfs -ls /spark-history/
> Found 8 items
> -rwxrwx--- 3 hrt_qa hadoop 28793 2016-07-16 15:56
> /spark-history/application_1468678823755_0049
> -rwxrwx--- 3 hrt_qa hadoop 28763 2016-07-16 16:24
> /spark-history/application_1468678823755_0061
> -rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59
> /spark-history/application_1468686376753_0041_1
> -rwxrwx--- 3 hrt_qa hadoop 58841982 2016-07-16 19:11
> /spark-history/application_1468686376753_0043
> -rwxrwx--- 3 hive hadoop 5823 2016-07-16 11:38
> /spark-history/local-1468666932940
> -rwxrwx--- 3 hive hadoop 5757 2016-07-16 22:44
> /spark-history/local-1468669677840.inprogress
> -rwxrwx--- 3 hrt_qa had
[jira] [Updated] (SPARK-16595) Spark History server Rest Api gives Application not found error
[
https://issues.apache.org/jira/browse/SPARK-16595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-16595:
---
Description:
Scenario:
* Start SparkPi application in Spark1 using yarn-cluster mode
(application_1468686376753_0041)
* After application finishes validate application exists in respective Spark
History server.
{code}
Error loading url
http://xx.xx.xx.xx:18080/api/v1/applications/application_1468686376753_0041/1/executors
HTTP Code: 404
HTTP Data: no such app: application_1468686376753_0041{code}
{code:title=spark HS log}
16/07/16 15:55:29 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049.inprogress
16/07/16 15:56:20 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049
16/07/16 16:23:14 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061.inprogress
16/07/16 16:24:14 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061
16/07/16 17:42:32 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553.inprogress
16/07/16 17:43:22 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553
16/07/16 17:43:44 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376.inprogress
16/07/16 17:44:34 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376
16/07/16 18:53:10 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0041_1.inprogress
16/07/16 19:03:26 INFO PackagesResourceConfig: Scanning for root resource and
provider classes in the packages:
org.apache.spark.status.api.v1
16/07/16 19:03:35 INFO ScanningResourceConfig: Root resource classes found:
class org.apache.spark.status.api.v1.ApiRootResource
16/07/16 19:03:35 INFO ScanningResourceConfig: Provider classes found:
class org.apache.spark.status.api.v1.JacksonMessageWriter
16/07/16 19:03:35 INFO WebApplicationImpl: Initiating Jersey application,
version 'Jersey: 1.9 09/02/2011 11:17 AM'
16/07/16 19:03:36 INFO SecurityManager: Changing view acls to: spark
16/07/16 19:03:36 INFO SecurityManager: Changing modify acls to: spark
16/07/16 19:03:36 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(spark); users with
modify permissions: Set(spark)
16/07/16 19:03:36 INFO ApplicationCache: Failed to load application attempt
application_1468686376753_0041/Some(1)
16/07/16 19:04:21 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043.inprogress
16/07/16 19:12:02 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
16/07/16 19:16:11 INFO SecurityManager: Changing view acls to: spark
16/07/16 19:16:11 INFO SecurityManager: Changing modify acls to: spark
16/07/16 19:16:11 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(spark); users with
modify permissions: Set(spark)
16/07/16 19:16:11 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
16/07/16 19:16:22 INFO SecurityManager: Changing acls enabled to: false
16/07/16 19:16:22 INFO SecurityManager: Changing admin acls to:
16/07/16 19:16:22 INFO SecurityManager: Changing view acls to: hrt_qa{code}
{code}
hdfs@xxx:/var/log/spark$ hdfs dfs -ls /spark-history/
Found 8 items
-rwxrwx--- 3 hrt_qa hadoop 28793 2016-07-16 15:56
/spark-history/application_1468678823755_0049
-rwxrwx--- 3 hrt_qa hadoop 28763 2016-07-16 16:24
/spark-history/application_1468678823755_0061
-rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59
/spark-history/application_1468686376753_0041_1
-rwxrwx--- 3 hrt_qa hadoop 58841982 2016-07-16 19:11
/spark-history/application_1468686376753_0043
-rwxrwx--- 3 hive hadoop 5823 2016-07-16 11:38
/spark-history/local-1468666932940
-rwxrwx--- 3 hive hadoop 5757 2016-07-16 22:44
/spark-history/local-1468669677840.inprogress
-rwxrwx--- 3 hrt_qa hadoop 484113 2016-07-16 17:43
/spark-history/local-1468690940553
-rwxrwx--- 3 hrt_qa hadoop 57747 2016-07-16 17:44
/spark-history/local-1468691017376
hdfs@xxx:/var/log/spark$ hdfs dfs -ls
/spark-history/application_1468686376753_0041_1
-rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59
/spark-history/application_1468686376753_0041_1{code}
was:
Scenario:
* Start SparkPi application in Spark1 (application_1468686376753_0041) and
* After application finishes validate application exists in respective Spark
History se
[jira] [Updated] (SPARK-16595) Spark History server Rest Api gives Application not found error for yarn-cluster mode
[
https://issues.apache.org/jira/browse/SPARK-16595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-16595:
---
Summary: Spark History server Rest Api gives Application not found error
for yarn-cluster mode (was: Spark History server Rest Api gives Application
not found error)
> Spark History server Rest Api gives Application not found error for
> yarn-cluster mode
> -
>
> Key: SPARK-16595
> URL: https://issues.apache.org/jira/browse/SPARK-16595
> Project: Spark
> Issue Type: Bug
>Affects Versions: 1.6.1
>Reporter: Yesha Vora
>
> Scenario:
> * Start SparkPi application in Spark1 using yarn-cluster mode
> (application_1468686376753_0041)
> * After application finishes validate application exists in respective Spark
> History server.
> {code}
> Error loading url
> http://xx.xx.xx.xx:18080/api/v1/applications/application_1468686376753_0041/1/executors
> HTTP Code: 404
> HTTP Data: no such app: application_1468686376753_0041{code}
> {code:title=spark HS log}
> 16/07/16 15:55:29 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049.inprogress
> 16/07/16 15:56:20 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049
> 16/07/16 16:23:14 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061.inprogress
> 16/07/16 16:24:14 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061
> 16/07/16 17:42:32 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553.inprogress
> 16/07/16 17:43:22 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553
> 16/07/16 17:43:44 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376.inprogress
> 16/07/16 17:44:34 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376
> 16/07/16 18:53:10 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0041_1.inprogress
> 16/07/16 19:03:26 INFO PackagesResourceConfig: Scanning for root resource and
> provider classes in the packages:
> org.apache.spark.status.api.v1
> 16/07/16 19:03:35 INFO ScanningResourceConfig: Root resource classes found:
> class org.apache.spark.status.api.v1.ApiRootResource
> 16/07/16 19:03:35 INFO ScanningResourceConfig: Provider classes found:
> class org.apache.spark.status.api.v1.JacksonMessageWriter
> 16/07/16 19:03:35 INFO WebApplicationImpl: Initiating Jersey application,
> version 'Jersey: 1.9 09/02/2011 11:17 AM'
> 16/07/16 19:03:36 INFO SecurityManager: Changing view acls to: spark
> 16/07/16 19:03:36 INFO SecurityManager: Changing modify acls to: spark
> 16/07/16 19:03:36 INFO SecurityManager: SecurityManager: authentication
> disabled; ui acls disabled; users with view permissions: Set(spark); users
> with modify permissions: Set(spark)
> 16/07/16 19:03:36 INFO ApplicationCache: Failed to load application attempt
> application_1468686376753_0041/Some(1)
> 16/07/16 19:04:21 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043.inprogress
> 16/07/16 19:12:02 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
> 16/07/16 19:16:11 INFO SecurityManager: Changing view acls to: spark
> 16/07/16 19:16:11 INFO SecurityManager: Changing modify acls to: spark
> 16/07/16 19:16:11 INFO SecurityManager: SecurityManager: authentication
> disabled; ui acls disabled; users with view permissions: Set(spark); users
> with modify permissions: Set(spark)
> 16/07/16 19:16:11 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
> 16/07/16 19:16:22 INFO SecurityManager: Changing acls enabled to: false
> 16/07/16 19:16:22 INFO SecurityManager: Changing admin acls to:
> 16/07/16 19:16:22 INFO SecurityManager: Changing view acls to: hrt_qa{code}
> {code}
> hdfs@xxx:/var/log/spark$ hdfs dfs -ls /spark-history/
> Found 8 items
> -rwxrwx--- 3 hrt_qa hadoop 28793 2016-07-16 15:56
> /spark-history/application_1468678823755_0049
> -rwxrwx--- 3 hrt_qa hadoop 28763 2016-07-16 16:24
> /spark-history/application_1468678823755_0061
> -rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59
> /spark-history/application_1468686376753_0041_1
> -rwxrwx--- 3 hrt_qa hadoop 58841982 2016-07-16 19:11
> /spark-history/application_1468686376753_0043
> -rwxrwx--- 3 hive hadoop 5823 2016-07-16 11:38
>
[jira] [Created] (SPARK-16595) Spark History server Rest Api gives Application not found error
Yesha Vora created SPARK-16595:
--
Summary: Spark History server Rest Api gives Application not found
error
Key: SPARK-16595
URL: https://issues.apache.org/jira/browse/SPARK-16595
Project: Spark
Issue Type: Bug
Affects Versions: 1.6.1
Reporter: Yesha Vora
Scenario:
* Start SparkPi application in Spark1 (application_1468686376753_0041) and
* After application finishes validate application exists in respective Spark
History server.
{code}
Error loading url
http://xx.xx.xx.xx:18080/api/v1/applications/application_1468686376753_0041/1/executors
HTTP Code: 404
HTTP Data: no such app: application_1468686376753_0041{code}
{code:title=spark HS log}
16/07/16 15:55:29 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049.inprogress
16/07/16 15:56:20 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049
16/07/16 16:23:14 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061.inprogress
16/07/16 16:24:14 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061
16/07/16 17:42:32 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553.inprogress
16/07/16 17:43:22 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553
16/07/16 17:43:44 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376.inprogress
16/07/16 17:44:34 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376
16/07/16 18:53:10 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0041_1.inprogress
16/07/16 19:03:26 INFO PackagesResourceConfig: Scanning for root resource and
provider classes in the packages:
org.apache.spark.status.api.v1
16/07/16 19:03:35 INFO ScanningResourceConfig: Root resource classes found:
class org.apache.spark.status.api.v1.ApiRootResource
16/07/16 19:03:35 INFO ScanningResourceConfig: Provider classes found:
class org.apache.spark.status.api.v1.JacksonMessageWriter
16/07/16 19:03:35 INFO WebApplicationImpl: Initiating Jersey application,
version 'Jersey: 1.9 09/02/2011 11:17 AM'
16/07/16 19:03:36 INFO SecurityManager: Changing view acls to: spark
16/07/16 19:03:36 INFO SecurityManager: Changing modify acls to: spark
16/07/16 19:03:36 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(spark); users with
modify permissions: Set(spark)
16/07/16 19:03:36 INFO ApplicationCache: Failed to load application attempt
application_1468686376753_0041/Some(1)
16/07/16 19:04:21 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043.inprogress
16/07/16 19:12:02 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
16/07/16 19:16:11 INFO SecurityManager: Changing view acls to: spark
16/07/16 19:16:11 INFO SecurityManager: Changing modify acls to: spark
16/07/16 19:16:11 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(spark); users with
modify permissions: Set(spark)
16/07/16 19:16:11 INFO FsHistoryProvider: Replaying log path:
hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
16/07/16 19:16:22 INFO SecurityManager: Changing acls enabled to: false
16/07/16 19:16:22 INFO SecurityManager: Changing admin acls to:
16/07/16 19:16:22 INFO SecurityManager: Changing view acls to: hrt_qa{code}
{code}
hdfs@xxx:/var/log/spark$ hdfs dfs -ls /spark-history/
Found 8 items
-rwxrwx--- 3 hrt_qa hadoop 28793 2016-07-16 15:56
/spark-history/application_1468678823755_0049
-rwxrwx--- 3 hrt_qa hadoop 28763 2016-07-16 16:24
/spark-history/application_1468678823755_0061
-rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59
/spark-history/application_1468686376753_0041_1
-rwxrwx--- 3 hrt_qa hadoop 58841982 2016-07-16 19:11
/spark-history/application_1468686376753_0043
-rwxrwx--- 3 hive hadoop 5823 2016-07-16 11:38
/spark-history/local-1468666932940
-rwxrwx--- 3 hive hadoop 5757 2016-07-16 22:44
/spark-history/local-1468669677840.inprogress
-rwxrwx--- 3 hrt_qa hadoop 484113 2016-07-16 17:43
/spark-history/local-1468690940553
-rwxrwx--- 3 hrt_qa hadoop 57747 2016-07-16 17:44
/spark-history/local-1468691017376
hdfs@xxx:/var/log/spark$ hdfs dfs -ls
/spark-history/application_1468686376753_0041_1
-rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59
/spark-history/application_1468686376753_0041_1{code}
--
This message was sent by Atlas
[jira] [Updated] (SPARK-16541) SparkTC application could not shutdown successfully
[
https://issues.apache.org/jira/browse/SPARK-16541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-16541:
---
Description:
SparkTC application in yarn-client mode was stuck at 10% progress.
{code} spark-submit --class org.apache.spark.examples.SparkTC --master
yarn-client spark-examples-assembly_*.jar {code}
It seems like SparkTC application tasks finished and printed "TC has 6254
edges.". after that while shutting down, spark application kept getting "ERROR
LiveListenerBus: SparkListenerBus has already stopped! Dropping event
SparkListenerExecutorMetricsUpdate"
{code}
16/07/13 08:43:37 INFO DAGScheduler: ResultStage 283 (count at
SparkTC.scala:71) finished in 42.357 s
16/07/13 08:43:37 INFO DAGScheduler: Job 13 finished: count at
SparkTC.scala:71, took 43.137408 s
TC has 6254 edges.
16/07/13 08:43:37 INFO ServerConnector: Stopped
ServerConnector@5e0054a2{HTTP/1.1}{0.0.0.0:4040}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@7350a22{/stages/stage/kill,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@54d56a49{/api,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@2fd52e57{/,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@7737ff3{/static,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@499d9067{/executors/threadDump/json,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@40d0c2af{/executors/threadDump,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@44ce4013{/executors/json,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@59c9a28a{/executors,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@2e784443{/environment/json,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@10240ba4{/environment,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@4ee2dd22{/storage/rdd/json,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@79ab14cd{/storage/rdd,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@731d1285{/storage/json,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@72e46ea8{/storage,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@266dcdd5{/stages/pool/json,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@17ee6dd9{/stages/pool,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@717867ea{/stages/stage/json,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@74aaadcc{/stages/stage,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@14f35a42{/stages/json,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@27ec74f8{/stages,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@148ad9f9{/jobs/job/json,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@14445e4c{/jobs/job,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@6d1557ff{/jobs/json,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@aca62b1{/jobs,null,UNAVAILABLE}
16/07/13 08:43:37 INFO SparkUI: Stopped Spark web UI at http://xx.xx.xx:4040
16/07/13 08:43:37 ERROR LiveListenerBus: SparkListenerBus has already stopped!
Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray())
16/07/13 08:43:37 ERROR LiveListenerBus: SparkListenerBus has already stopped!
Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray())
16/07/13 08:43:47 ERROR LiveListenerBus: SparkListenerBus has already stopped!
Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray())
16/07/13 08:43:47 ERROR LiveListenerBus: SparkListenerBus has already stopped!
Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray())
16/07/13 08:43:57 ERROR LiveListenerBus: SparkListenerBus has already stopped!
Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray())
16/07/13 08:43:57 ERROR LiveListenerBus: SparkListenerBus has already stopped!
Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray())
16/07/13 08:44:07 ERROR LiveListenerBus: SparkListenerBus has already stopped!
Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray())
16/07/13 08:44:07 ERROR LiveListenerBus: SparkListenerBus
[jira] [Commented] (SPARK-16541) SparkTC application could not shutdown successfully
[
https://issues.apache.org/jira/browse/SPARK-16541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376551#comment-15376551
]
Yesha Vora commented on SPARK-16541:
[~sowen], Stack Trace updated.
> SparkTC application could not shutdown successfully
>
>
> Key: SPARK-16541
> URL: https://issues.apache.org/jira/browse/SPARK-16541
> Project: Spark
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Yesha Vora
>
> SparkTC application in yarn-client mode was stuck at 10% progress.
> {code} spark-submit --class org.apache.spark.examples.SparkTC --master
> yarn-client spark-examples-assembly_*.jar {code}
> It seems like SparkTC application tasks finished and printed "TC has 6254
> edges.". after that while shutting down, spark application kept getting
> "ERROR LiveListenerBus: SparkListenerBus has already stopped! Dropping event
> SparkListenerExecutorMetricsUpdate"
> {code}
> 16/07/13 08:43:37 INFO DAGScheduler: ResultStage 283 (count at
> SparkTC.scala:71) finished in 42.357 s
> 16/07/13 08:43:37 INFO DAGScheduler: Job 13 finished: count at
> SparkTC.scala:71, took 43.137408 s
> TC has 6254 edges.
> 16/07/13 08:43:37 INFO ServerConnector: Stopped
> ServerConnector@5e0054a2{HTTP/1.1}{0.0.0.0:4040}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@7350a22{/stages/stage/kill,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@54d56a49{/api,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@2fd52e57{/,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@7737ff3{/static,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@499d9067{/executors/threadDump/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@40d0c2af{/executors/threadDump,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@44ce4013{/executors/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@59c9a28a{/executors,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@2e784443{/environment/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@10240ba4{/environment,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@4ee2dd22{/storage/rdd/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@79ab14cd{/storage/rdd,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@731d1285{/storage/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@72e46ea8{/storage,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@266dcdd5{/stages/pool/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@17ee6dd9{/stages/pool,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@717867ea{/stages/stage/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@74aaadcc{/stages/stage,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@14f35a42{/stages/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@27ec74f8{/stages,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@148ad9f9{/jobs/job/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@14445e4c{/jobs/job,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@6d1557ff{/jobs/json,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO ContextHandler: Stopped
> o.s.j.s.ServletContextHandler@aca62b1{/jobs,null,UNAVAILABLE}
> 16/07/13 08:43:37 INFO SparkUI: Stopped Spark web UI at http://xx.xx.xx:4040
> 16/07/13 08:43:37 ERROR LiveListenerBus: SparkListenerBus has already
> stopped! Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray())
> 16/07/13 08:43:37 ERROR LiveListenerBus: SparkListenerBus has already
> stopped! Dropping event SparkListenerExecutorMetricsUpdate(1,WrappedArray())
> 16/07/13 08:43:47 ERROR LiveListenerBus: SparkListenerBus has already
> stopped! Dropping event SparkListenerExecutorMetricsUpdate(2,WrappedArray())
> 16/07/13 08:43:47 ERROR LiveListenerBus: SparkListenerBus has already
> stopped! Dropping even
[jira] [Created] (SPARK-16541) SparkTC application could not shutdown successfully
Yesha Vora created SPARK-16541:
--
Summary: SparkTC application could not shutdown successfully
Key: SPARK-16541
URL: https://issues.apache.org/jira/browse/SPARK-16541
Project: Spark
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yesha Vora
SparkTC application in yarn-client mode was stuck at 10% progress.
{code} spark-submit --class org.apache.spark.examples.SparkTC --master
yarn-client spark-examples-assembly_*.jar {code}
It seems like SparkTC application tasks finished and printed "TC has 6254
edges.". after that while shutting down, spark application kept getting "ERROR
LiveListenerBus: SparkListenerBus has already stopped! Dropping event
SparkListenerExecutorMetricsUpdate"
{code}
16/07/13 08:43:37 INFO DAGScheduler: ResultStage 283 (count at
SparkTC.scala:71) finished in 42.357 s
16/07/13 08:43:37 INFO DAGScheduler: Job 13 finished: count at
SparkTC.scala:71, took 43.137408 s
TC has 6254 edges.
16/07/13 08:43:37 INFO ServerConnector: Stopped
ServerConnector@5e0054a2{HTTP/1.1}{0.0.0.0:4040}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@7350a22{/stages/stage/kill,null,UNAVAILABLE}
16/07/13 08:43:37 INFO ContextHandler: Stopped
o.s.j.s.ServletContextHandler@54d56a49{/api,null,UNAVAILABLE}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-16451) Spark-shell / pyspark should finish gracefully when "SaslException: GSS initiate failed" is hit
Yesha Vora created SPARK-16451:
--
Summary: Spark-shell / pyspark should finish gracefully when
"SaslException: GSS initiate failed" is hit
Key: SPARK-16451
URL: https://issues.apache.org/jira/browse/SPARK-16451
Project: Spark
Issue Type: Bug
Affects Versions: 1.6.1
Reporter: Yesha Vora
Steps to reproduce: (secure cluster)
* kdestroy
* spark-shell --master yarn-client
If no valid keytab is set while running spark-shell/pyspark, the spark client
never exits. It keep printing below error messages.
spark-client should call shutdown hook immediately and exit with proper error
code.
Currently, user need to explicitly shutdown process. (using cntrl+c)
{code}
16/07/08 20:53:10 WARN Client: Exception encountered while connecting to the
server :
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException:
No valid credentials provided (Mechanism level: Failed to find any Kerberos
tgt)]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
at
org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:413)
at
org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:595)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:397)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:761)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:757)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:756)
at org.apache.hadoop.ipc.Client$Connection.access$3200(Client.java:397)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1617)
at org.apache.hadoop.ipc.Client.call(Client.java:1448)
at org.apache.hadoop.ipc.Client.call(Client.java:1395)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
at com.sun.proxy.$Proxy25.getFileInfo(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:816)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
at com.sun.proxy.$Proxy26.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2151)
at
org.apache.hadoop.hdfs.DistributedFileSystem$25.doCall(DistributedFileSystem.java:1408)
at
org.apache.hadoop.hdfs.DistributedFileSystem$25.doCall(DistributedFileSystem.java:1404)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1404)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1437)
at
org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter.(FileSystemTimelineWriter.java:124)
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.createTimelineWriter(TimelineClientImpl.java:316)
at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceStart(TimelineClientImpl.java:308)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:194)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:127)
at
org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
at
org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)
at org.apache.spark.SparkContext.(SparkContext.scala:530)
at
org.apache.spark.repl.SparkILoop.createSparkContext(SparkILoop.scala:1017)
at $line3.$read$$iwC$$iwC.(:15)
at $line3.$read$$iwC.(:24)
at $line3.$read.(:26)
at $line3.$read$.(:30)
at $line3.$read$.()
at $line3.$eval$.(:7)
at $line3.$eval$.()
at $line3.$eval.$print()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodA
[jira] [Resolved] (SPARK-15847) DecisionTreeRunner example stucks with "NoClassDefFoundError: org/apache/avro/generic/GenericRecord"
[
https://issues.apache.org/jira/browse/SPARK-15847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora resolved SPARK-15847.
Resolution: Cannot Reproduce
> DecisionTreeRunner example stucks with "NoClassDefFoundError:
> org/apache/avro/generic/GenericRecord"
>
>
> Key: SPARK-15847
> URL: https://issues.apache.org/jira/browse/SPARK-15847
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 2.0.0
>Reporter: Yesha Vora
>
> In Spark-2, DecisionTreeRunner example hangs with "NoClassDefFoundError:
> org/apache/avro/generic/GenericRecord".
> The same application passes in yarn-cluster mode. I'm hitting this issue with
> yarn-client mode only.
> {code}
> spark-submit --class org.apache.spark.examples.mllib.DecisionTreeRunner
> --master yarn-client --jars hadoop-lzo-*.jar /xxx/lib/spark-examples_*jar
> /tmp/sparkMLLInput/sample_libsvm_data.txt{code}
> {code}
> 16/05/27 02:37:50 INFO SparkContext: Starting job: countByValue at
> DecisionTreeRunner.scala:185
> Exception in thread "dag-scheduler-event-loop"
> java.lang.NoClassDefFoundError: org/apache/avro/generic/GenericRecord
> at
> org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:112)
> at
> org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:274)
> at
> org.apache.spark.serializer.KryoSerializerInstance.(KryoSerializer.scala:259)
> at
> org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:175)
> at
> org.apache.spark.serializer.KryoSerializer.supportsRelocationOfSerializedObjects$lzycompute(KryoSerializer.scala:182)
> at
> org.apache.spark.serializer.KryoSerializer.supportsRelocationOfSerializedObjects(KryoSerializer.scala:178)
> at
> org.apache.spark.shuffle.sort.SortShuffleManager$.canUseSerializedShuffle(SortShuffleManager.scala:187)
> at
> org.apache.spark.shuffle.sort.SortShuffleManager.registerShuffle(SortShuffleManager.scala:99)
> at org.apache.spark.ShuffleDependency.(Dependency.scala:90)
> at org.apache.spark.rdd.ShuffledRDD.getDependencies(ShuffledRDD.scala:91)
> at org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:234)
> at org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:232)
> at scala.Option.getOrElse(Option.scala:121)
> at org.apache.spark.rdd.RDD.dependencies(RDD.scala:232)
> at org.apache.spark.scheduler.DAGScheduler.visit$1(DAGScheduler.scala:391)
> at
> org.apache.spark.scheduler.DAGScheduler.getParentStages(DAGScheduler.scala:403)
> at
> org.apache.spark.scheduler.DAGScheduler.getParentStagesAndId(DAGScheduler.scala:304)
> at
> org.apache.spark.scheduler.DAGScheduler.newResultStage(DAGScheduler.scala:339)
> at
> org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:849)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1626)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1618)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1607)
> at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.avro.generic.GenericRecord
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> .. 23 more{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-16314) Spark application got stuck when NM running executor is restarted
[
https://issues.apache.org/jira/browse/SPARK-16314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356401#comment-15356401
]
Yesha Vora commented on SPARK-16314:
Thanks [~jerryshao] for analysis.
{code}
Looking though the log, I think we're running into some RPC timeout and retry
problems. In this scenario NM recovery is enabled:
1. we will kill and restart the NM, so this will run into a race condition
where container is allocated and executor is starting to connect to external
shuffle service, in this time if NM is failed, executor will be failed (cannot
connect to external shuffle service).
2. Once executor is exited, driver will issue RPC requests to ask AM the reason
about failure, in this situation failed executors are in the zombie status,
which means driver will still keep the metadata of these executor, only when AM
report back the results driver will clean the zombie executors. But in the NM
failed situation, AM cannot get the failed container state until RPC timeout
(120s), also timed out RPC will be retried (again wait until 120s timeout).
3. In the meantime If more than 3 executors are failed due to this issue AM and
driver will be exited. At this time if NM is restarted, it will report failed
containers to AM and AM will send RemoveExecutor to driver, at this time driver
is already exited, so this message never be delivered, wait until timeout
(120s) and retry.
So this cumulative timeout will hang the application exiting and delay
reattempt of this application, that's why we saw the application is hang.
I think in this test, we're running into the corner case.
{code}
> Spark application got stuck when NM running executor is restarted
> -
>
> Key: SPARK-16314
> URL: https://issues.apache.org/jira/browse/SPARK-16314
> Project: Spark
> Issue Type: Bug
>Affects Versions: 1.6.1
>Reporter: Yesha Vora
>
> Spark Application hangs if Nodemanager running executor is stopped.
> * start LogQuery application
> * This application starts 2 executors. Each in different nodes.
> * restart one of the nodemanagers.
> The application stays at 10% progress till 12 minutes.
> Expected behavior: Application should either pass or fail. It should not
> hang.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-16314) Spark application got stuck when NM running executor is restarted
Yesha Vora created SPARK-16314: -- Summary: Spark application got stuck when NM running executor is restarted Key: SPARK-16314 URL: https://issues.apache.org/jira/browse/SPARK-16314 Project: Spark Issue Type: Bug Affects Versions: 1.6.1 Reporter: Yesha Vora Spark Application hangs if Nodemanager running executor is stopped. * start LogQuery application * This application starts 2 executors. Each in different nodes. * restart one of the nodemanagers. The application stays at 10% progress till 12 minutes. Expected behavior: Application should either pass or fail. It should not hang. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-15854) Spark History server gets null pointer exception
[
https://issues.apache.org/jira/browse/SPARK-15854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora resolved SPARK-15854.
Resolution: Cannot Reproduce
> Spark History server gets null pointer exception
>
>
> Key: SPARK-15854
> URL: https://issues.apache.org/jira/browse/SPARK-15854
> Project: Spark
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Yesha Vora
>
> In Spark2, Spark-History Server is configured to FSHistoryProvider.
> Spark HS does not show any finished/running applications and gets Null
> pointer exception.
> {code}
> 16/06/03 23:06:40 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx:8020/spark2-history/application_1464912457462_0002.inprogress
> 16/06/03 23:06:50 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx:8020/spark2-history/application_1464912457462_0002
> 16/06/03 23:08:27 WARN ServletHandler: Error for /api/v1/applications
> java.lang.NoSuchMethodError:
> javax.ws.rs.core.Application.getProperties()Ljava/util/Map;
> at
> org.glassfish.jersey.server.ApplicationHandler.(ApplicationHandler.java:331)
> at
> org.glassfish.jersey.servlet.WebComponent.(WebComponent.java:392)
> at
> org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:177)
> at
> org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:369)
> at javax.servlet.GenericServlet.init(GenericServlet.java:244)
> at
> org.spark_project.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:616)
> at
> org.spark_project.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:472)
> at
> org.spark_project.jetty.servlet.ServletHolder.ensureInstance(ServletHolder.java:767)
> at
> org.spark_project.jetty.servlet.ServletHolder.prepare(ServletHolder.java:752)
> at
> org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> at
> org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.spark_project.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:479)
> at
> org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at
> org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.spark_project.jetty.server.Server.handle(Server.java:499)
> at
> org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:311)
> at
> org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> org.spark_project.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
> at
> org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.spark_project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
> 16/06/03 23:08:33 WARN ServletHandler: /api/v1/applications
> java.lang.NullPointerException
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
> at
> org.spark_project.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
> at
> org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
> at
> org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.spark_project.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:479)
> at
> org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at
> org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.spark_project.jetty.server.Server.handle(Server.java:499)
> at
> org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:311)
> at
> org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>
[jira] [Commented] (SPARK-15955) Failed Spark application returns with exitcode equals to zero
[
https://issues.apache.org/jira/browse/SPARK-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15347360#comment-15347360
]
Yesha Vora commented on SPARK-15955:
[~sowen], I'm checking exit code of the process which started the application.
[~tgraves]. This issue happens with yarn-client and yarn-cluster mode both.
> Failed Spark application returns with exitcode equals to zero
> -
>
> Key: SPARK-15955
> URL: https://issues.apache.org/jira/browse/SPARK-15955
> Project: Spark
> Issue Type: Bug
>Affects Versions: 1.6.1
>Reporter: Yesha Vora
>
> Scenario:
> * Set up cluster with wire-encryption enabled.
> * set 'spark.authenticate.enableSaslEncryption' = 'false' and
> 'spark.shuffle.service.enabled' :'true'
> * run sparkPi application.
> {code}
> client token: Token { kind: YARN_CLIENT_TOKEN, service: }
> diagnostics: Max number of executor failures (3) reached
> ApplicationMaster host: xx.xx.xx.xxx
> ApplicationMaster RPC port: 0
> queue: default
> start time: 1465941051976
> final status: FAILED
> tracking URL: https://xx.xx.xx.xxx:8090/proxy/application_1465925772890_0016/
> user: hrt_qa
> Exception in thread "main" org.apache.spark.SparkException: Application
> application_1465925772890_0016 finished with failed status
> at org.apache.spark.deploy.yarn.Client.run(Client.scala:1092)
> at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1139)
> at org.apache.spark.deploy.yarn.Client.main(Client.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> INFO ShutdownHookManager: Shutdown hook called{code}
> This spark application exits with exitcode = 0. Failed application should not
> return with exitcode = 0
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-16085) Spark stand-alone ui redirects to RM application master UI for yarn-client mode
[
https://issues.apache.org/jira/browse/SPARK-16085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-16085:
---
Affects Version/s: 1.6.1
> Spark stand-alone ui redirects to RM application master UI for yarn-client
> mode
> ---
>
> Key: SPARK-16085
> URL: https://issues.apache.org/jira/browse/SPARK-16085
> Project: Spark
> Issue Type: Bug
>Affects Versions: 1.6.1
>Reporter: Yesha Vora
>
> Spark UI in yarn-client mode redirects to Application Master UI.
> Run sparkPi application in yarn-client mode.
> When application is running , SparkUI is shown as " http://:4040"
> {code}
> 16/06/17 01:13:46 INFO AbstractConnector: Started
> SelectChannelConnector@0.0.0.0:4040
> 16/06/17 01:13:46 INFO Utils: Successfully started service 'SparkUI' on port
> 4040.
> 16/06/17 01:13:46 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at
> http://xx.xx.xx.xxx:4040
> {code}
> The spark UI should be accessible at " http://:4040". It should not be
> redirect to Application master ui.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-16085) Spark stand-alone ui redirects to RM application master UI for yarn-client mode
Yesha Vora created SPARK-16085:
--
Summary: Spark stand-alone ui redirects to RM application master
UI for yarn-client mode
Key: SPARK-16085
URL: https://issues.apache.org/jira/browse/SPARK-16085
Project: Spark
Issue Type: Bug
Reporter: Yesha Vora
Spark UI in yarn-client mode redirects to Application Master UI.
Run sparkPi application in yarn-client mode.
When application is running , SparkUI is shown as " http://:4040"
{code}
16/06/17 01:13:46 INFO AbstractConnector: Started
SelectChannelConnector@0.0.0.0:4040
16/06/17 01:13:46 INFO Utils: Successfully started service 'SparkUI' on port
4040.
16/06/17 01:13:46 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at
http://xx.xx.xx.xxx:4040
{code}
The spark UI should be accessible at " http://:4040". It should not be
redirect to Application master ui.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15955) Failed Spark application returns with exitcode equals to zero
[
https://issues.apache.org/jira/browse/SPARK-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-15955:
---
Summary: Failed Spark application returns with exitcode equals to zero
(was: Failed Spark application returns with client console equals zero)
> Failed Spark application returns with exitcode equals to zero
> -
>
> Key: SPARK-15955
> URL: https://issues.apache.org/jira/browse/SPARK-15955
> Project: Spark
> Issue Type: Bug
>Affects Versions: 1.6.1
>Reporter: Yesha Vora
>
> Scenario:
> * Set up cluster with wire-encryption enabled.
> * set 'spark.authenticate.enableSaslEncryption' = 'false' and
> 'spark.shuffle.service.enabled' :'true'
> * run sparkPi application.
> {code}
> client token: Token { kind: YARN_CLIENT_TOKEN, service: }
> diagnostics: Max number of executor failures (3) reached
> ApplicationMaster host: xx.xx.xx.xxx
> ApplicationMaster RPC port: 0
> queue: default
> start time: 1465941051976
> final status: FAILED
> tracking URL: https://xx.xx.xx.xxx:8090/proxy/application_1465925772890_0016/
> user: hrt_qa
> Exception in thread "main" org.apache.spark.SparkException: Application
> application_1465925772890_0016 finished with failed status
> at org.apache.spark.deploy.yarn.Client.run(Client.scala:1092)
> at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1139)
> at org.apache.spark.deploy.yarn.Client.main(Client.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> INFO ShutdownHookManager: Shutdown hook called{code}
> This spark application exits with exitcode = 0. Failed application should not
> return with exitcode = 0
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-15955) Failed Spark application returns with client console equals zero
Yesha Vora created SPARK-15955:
--
Summary: Failed Spark application returns with client console
equals zero
Key: SPARK-15955
URL: https://issues.apache.org/jira/browse/SPARK-15955
Project: Spark
Issue Type: Bug
Affects Versions: 1.6.1
Reporter: Yesha Vora
Scenario:
* Set up cluster with wire-encryption enabled.
* set 'spark.authenticate.enableSaslEncryption' = 'false' and
'spark.shuffle.service.enabled' :'true'
* run sparkPi application.
{code}
client token: Token { kind: YARN_CLIENT_TOKEN, service: }
diagnostics: Max number of executor failures (3) reached
ApplicationMaster host: xx.xx.xx.xxx
ApplicationMaster RPC port: 0
queue: default
start time: 1465941051976
final status: FAILED
tracking URL: https://xx.xx.xx.xxx:8090/proxy/application_1465925772890_0016/
user: hrt_qa
Exception in thread "main" org.apache.spark.SparkException: Application
application_1465925772890_0016 finished with failed status
at org.apache.spark.deploy.yarn.Client.run(Client.scala:1092)
at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1139)
at org.apache.spark.deploy.yarn.Client.main(Client.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
INFO ShutdownHookManager: Shutdown hook called{code}
This spark application exits with exitcode = 0. Failed application should not
return with exitcode = 0
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-15923) Spark Application rest api returns "no such app: "
Yesha Vora created SPARK-15923:
--
Summary: Spark Application rest api returns "no such app: "
Key: SPARK-15923
URL: https://issues.apache.org/jira/browse/SPARK-15923
Project: Spark
Issue Type: Bug
Affects Versions: 1.6.1
Reporter: Yesha Vora
Env : secure cluster
Scenario:
* Run SparkPi application in yarn-client or yarn-cluster mode
* After application finishes, check Spark HS rest api to get details like jobs
/ executor etc.
{code}
http://:18080/api/v1/applications/application_1465778870517_0001/1/executors{code}
Rest api return HTTP Code: 404 and prints "HTTP Data: no such app:
application_1465778870517_0001"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-15855) dataframe.R example fails with "java.io.IOException: No input paths specified in job"
Yesha Vora created SPARK-15855:
--
Summary: dataframe.R example fails with "java.io.IOException: No
input paths specified in job"
Key: SPARK-15855
URL: https://issues.apache.org/jira/browse/SPARK-15855
Project: Spark
Issue Type: Bug
Components: Examples
Affects Versions: 1.6.1
Reporter: Yesha Vora
Steps:
* Install R on all nodes
* Run dataframe.R example.
The example fails in yarn-client and yarn-cluster mode both with below
mentioned error message.
This application fails to find people.json correctly. {{path <-
file.path(Sys.getenv("SPARK_HOME"), "examples/src/main/resources/people.json")}}
{code}
[xxx@xxx qa]$ sparkR --master yarn-client examples/src/main/r/dataframe.R
Loading required package: methods
Attaching package: ‘SparkR’
The following objects are masked from ‘package:stats’:
cov, filter, lag, na.omit, predict, sd, var
The following objects are masked from ‘package:base’:
colnames, colnames<-, intersect, rank, rbind, sample, subset,
summary, table, transform
16/05/24 22:08:21 INFO SparkContext: Running Spark version 1.6.1
16/05/24 22:08:21 WARN NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
16/05/24 22:08:22 INFO SecurityManager: Changing view acls to: hrt_qa
16/05/24 22:08:22 INFO SecurityManager: Changing modify acls to: hrt_qa
16/05/24 22:08:22 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(hrt_qa); users
with modify permissions: Set(hrt_qa)
16/05/24 22:08:22 INFO Utils: Successfully started service 'sparkDriver' on
port 35792.
16/05/24 22:08:23 INFO Slf4jLogger: Slf4jLogger started
16/05/24 22:08:23 INFO Remoting: Starting remoting
16/05/24 22:08:23 INFO Remoting: Remoting started; listening on addresses
:[akka.tcp://sparkdriveractorsys...@xx.xx.xx.xxx:49771]
16/05/24 22:08:23 INFO Utils: Successfully started service
'sparkDriverActorSystem' on port 49771.
16/05/24 22:08:23 INFO SparkEnv: Registering MapOutputTracker
16/05/24 22:08:23 INFO SparkEnv: Registering BlockManagerMaster
16/05/24 22:08:23 INFO DiskBlockManager: Created local directory at
/tmp/blockmgr-ffed73ad-3e67-4ae5-8734-9338136d3721
16/05/24 22:08:23 INFO MemoryStore: MemoryStore started with capacity 511.1 MB
16/05/24 22:08:24 INFO SparkEnv: Registering OutputCommitCoordinator
16/05/24 22:08:24 INFO Server: jetty-8.y.z-SNAPSHOT
16/05/24 22:08:24 INFO AbstractConnector: Started
SelectChannelConnector@0.0.0.0:4040
16/05/24 22:08:24 INFO Utils: Successfully started service 'SparkUI' on port
4040.
16/05/24 22:08:24 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at
http://xx.xx.xx.xxx:4040
spark.yarn.driver.memoryOverhead is set but does not apply in client mode.
16/05/24 22:08:25 INFO Client: Requesting a new application from cluster with 6
NodeManagers
16/05/24 22:08:25 INFO Client: Verifying our application has not requested more
than the maximum memory capability of the cluster (10240 MB per container)
16/05/24 22:08:25 INFO Client: Will allocate AM container, with 896 MB memory
including 384 MB overhead
16/05/24 22:08:25 INFO Client: Setting up container launch context for our AM
16/05/24 22:08:25 INFO Client: Setting up the launch environment for our AM
container
16/05/24 22:08:26 WARN DomainSocketFactory: The short-circuit local reads
feature cannot be used because libhadoop cannot be loaded.
16/05/24 22:08:26 INFO Client: Using the spark assembly jar on HDFS because you
are using HDP,
defaultSparkAssembly:hdfs://mycluster/hdp/apps/2.5.0.0-427/spark/spark-hdp-assembly.jar
16/05/24 22:08:26 INFO Client: Preparing resources for our AM container
16/05/24 22:08:26 INFO YarnSparkHadoopUtil: getting token for namenode:
hdfs://mycluster/user/hrt_qa/.sparkStaging/application_1463956206030_0003
16/05/24 22:08:26 INFO DFSClient: Created HDFS_DELEGATION_TOKEN token 187 for
hrt_qa on ha-hdfs:mycluster
16/05/24 22:08:28 INFO metastore: Trying to connect to metastore with URI
thrift://xxx:9083
16/05/24 22:08:28 INFO metastore: Connected to metastore.
16/05/24 22:08:28 INFO YarnSparkHadoopUtil: HBase class not found
java.lang.ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration
16/05/24 22:08:28 INFO Client: Using the spark assembly jar on HDFS because you
are using HDP,
defaultSparkAssembly:hdfs://mycluster/hdp/apps/2.5.0.0-427/spark/spark-hdp-assembly.jar
16/05/24 22:08:28 INFO Client: Source and destination file systems are the
same. Not copying
hdfs://mycluster/hdp/apps/2.5.0.0-427/spark/spark-hdp-assembly.jar
16/05/24 22:08:29 INFO Client: Uploading resource
file:/usr/hdp/current/spark-client/examples/src/main/r/dataframe.R ->
hdfs://mycluster/user/hrt_qa/.sparkStaging/application_1463956206030_0003/dataframe.R
16/05/24 22:08:29 INFO Client: Uploading resource
file:/grid/0/spark/R/lib/sparkr.zip#sp
[jira] [Created] (SPARK-15854) Spark History server gets null pointer exception
Yesha Vora created SPARK-15854:
--
Summary: Spark History server gets null pointer exception
Key: SPARK-15854
URL: https://issues.apache.org/jira/browse/SPARK-15854
Project: Spark
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Yesha Vora
In Spark2, Spark-History Server is configured to FSHistoryProvider.
Spark HS does not show any finished/running applications and gets Null pointer
exception.
{code}
16/06/03 23:06:40 INFO FsHistoryProvider: Replaying log path:
hdfs://xx:8020/spark2-history/application_1464912457462_0002.inprogress
16/06/03 23:06:50 INFO FsHistoryProvider: Replaying log path:
hdfs://xx:8020/spark2-history/application_1464912457462_0002
16/06/03 23:08:27 WARN ServletHandler: Error for /api/v1/applications
java.lang.NoSuchMethodError:
javax.ws.rs.core.Application.getProperties()Ljava/util/Map;
at
org.glassfish.jersey.server.ApplicationHandler.(ApplicationHandler.java:331)
at
org.glassfish.jersey.servlet.WebComponent.(WebComponent.java:392)
at
org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:177)
at
org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:369)
at javax.servlet.GenericServlet.init(GenericServlet.java:244)
at
org.spark_project.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:616)
at
org.spark_project.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:472)
at
org.spark_project.jetty.servlet.ServletHolder.ensureInstance(ServletHolder.java:767)
at
org.spark_project.jetty.servlet.ServletHolder.prepare(ServletHolder.java:752)
at
org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at
org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at
org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at
org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.spark_project.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:479)
at
org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at
org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.spark_project.jetty.server.Server.handle(Server.java:499)
at
org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at
org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at
org.spark_project.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at
org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at
org.spark_project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
16/06/03 23:08:33 WARN ServletHandler: /api/v1/applications
java.lang.NullPointerException
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
at
org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
at
org.spark_project.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
at
org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
at
org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at
org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at
org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at
org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.spark_project.jetty.servlets.gzip.GzipHandler.handle(GzipHandler.java:479)
at
org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at
org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.spark_project.jetty.server.Server.handle(Server.java:499)
at
org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at
org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at
org.spark_project.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at
org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at
org.spark_project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlas
[jira] [Updated] (SPARK-15830) Spark application should get hive tokens only when it is required
[
https://issues.apache.org/jira/browse/SPARK-15830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-15830:
---
Affects Version/s: 1.6.1
> Spark application should get hive tokens only when it is required
> -
>
> Key: SPARK-15830
> URL: https://issues.apache.org/jira/browse/SPARK-15830
> Project: Spark
> Issue Type: Improvement
>Affects Versions: 1.6.1
>Reporter: Yesha Vora
>
> Currently , All spark application try to get Hive tokens (Even if application
> does not use them) if Hive is installed on the cluster.
> Due to this practice, spark application which does not require Hive fails
> when Hive service (metastore) is down for some reason.
> Thus, spark should only try to get Hive tokens when required. It should not
> fetch hive token if it is not needed by application.
> Example : Spark Pi application does not perform any hive related actions. But
> Spark Pi application still fails if hive metastore service is down.
> {code}
> 16/06/08 01:18:42 INFO YarnSparkHadoopUtil: getting token for namenode:
> hdfs://xxx:8020/user/xx/.sparkStaging/application_1465347287950_0001
> 16/06/08 01:18:42 INFO DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for
> xx on xx.xx.xx.xxx:8020
> 16/06/08 01:18:43 INFO metastore: Trying to connect to metastore with URI
> thrift://xx.xx.xx.xxx:9090
> 16/06/08 01:18:43 WARN metastore: Failed to connect to the MetaStore Server...
> 16/06/08 01:18:43 INFO metastore: Waiting 5 seconds before next connection
> attempt.
> 16/06/08 01:18:48 INFO metastore: Trying to connect to metastore with URI
> thrift://xx.xx.xx.xxx:9090
> 16/06/08 01:18:48 WARN metastore: Failed to connect to the MetaStore Server...
> 16/06/08 01:18:48 INFO metastore: Waiting 5 seconds before next connection
> attempt.
> 16/06/08 01:18:53 INFO metastore: Trying to connect to metastore with URI
> thrift://xx.xx.xx.xxx:9090
> 16/06/08 01:18:53 WARN metastore: Failed to connect to the MetaStore Server...
> 16/06/08 01:18:53 INFO metastore: Waiting 5 seconds before next connection
> attempt.
> 16/06/08 01:18:59 WARN Hive: Failed to access metastore. This class should
> not accessed in runtime.
> org.apache.hadoop.hive.ql.metadata.Hive Exception : java.lang.Runtime
> Exception : Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1236)
> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
> at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:166)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15847) DecisionTreeRunner example stucks with "NoClassDefFoundError: org/apache/avro/generic/GenericRecord"
[
https://issues.apache.org/jira/browse/SPARK-15847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-15847:
---
Affects Version/s: 2.0.0
> DecisionTreeRunner example stucks with "NoClassDefFoundError:
> org/apache/avro/generic/GenericRecord"
>
>
> Key: SPARK-15847
> URL: https://issues.apache.org/jira/browse/SPARK-15847
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 2.0.0
>Reporter: Yesha Vora
>
> In Spark-2, DecisionTreeRunner example hangs with "NoClassDefFoundError:
> org/apache/avro/generic/GenericRecord".
> The same application passes in yarn-cluster mode. I'm hitting this issue with
> yarn-client mode only.
> {code}
> spark-submit --class org.apache.spark.examples.mllib.DecisionTreeRunner
> --master yarn-client --jars hadoop-lzo-*.jar /xxx/lib/spark-examples_*jar
> /tmp/sparkMLLInput/sample_libsvm_data.txt{code}
> {code}
> 16/05/27 02:37:50 INFO SparkContext: Starting job: countByValue at
> DecisionTreeRunner.scala:185
> Exception in thread "dag-scheduler-event-loop"
> java.lang.NoClassDefFoundError: org/apache/avro/generic/GenericRecord
> at
> org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:112)
> at
> org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:274)
> at
> org.apache.spark.serializer.KryoSerializerInstance.(KryoSerializer.scala:259)
> at
> org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:175)
> at
> org.apache.spark.serializer.KryoSerializer.supportsRelocationOfSerializedObjects$lzycompute(KryoSerializer.scala:182)
> at
> org.apache.spark.serializer.KryoSerializer.supportsRelocationOfSerializedObjects(KryoSerializer.scala:178)
> at
> org.apache.spark.shuffle.sort.SortShuffleManager$.canUseSerializedShuffle(SortShuffleManager.scala:187)
> at
> org.apache.spark.shuffle.sort.SortShuffleManager.registerShuffle(SortShuffleManager.scala:99)
> at org.apache.spark.ShuffleDependency.(Dependency.scala:90)
> at org.apache.spark.rdd.ShuffledRDD.getDependencies(ShuffledRDD.scala:91)
> at org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:234)
> at org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:232)
> at scala.Option.getOrElse(Option.scala:121)
> at org.apache.spark.rdd.RDD.dependencies(RDD.scala:232)
> at org.apache.spark.scheduler.DAGScheduler.visit$1(DAGScheduler.scala:391)
> at
> org.apache.spark.scheduler.DAGScheduler.getParentStages(DAGScheduler.scala:403)
> at
> org.apache.spark.scheduler.DAGScheduler.getParentStagesAndId(DAGScheduler.scala:304)
> at
> org.apache.spark.scheduler.DAGScheduler.newResultStage(DAGScheduler.scala:339)
> at
> org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:849)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1626)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1618)
> at
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1607)
> at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.avro.generic.GenericRecord
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> .. 23 more{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-15847) DecisionTreeRunner example stucks with "NoClassDefFoundError: org/apache/avro/generic/GenericRecord"
Yesha Vora created SPARK-15847:
--
Summary: DecisionTreeRunner example stucks with
"NoClassDefFoundError: org/apache/avro/generic/GenericRecord"
Key: SPARK-15847
URL: https://issues.apache.org/jira/browse/SPARK-15847
Project: Spark
Issue Type: Bug
Components: Spark Core
Reporter: Yesha Vora
In Spark-2, DecisionTreeRunner example hangs with "NoClassDefFoundError:
org/apache/avro/generic/GenericRecord".
The same application passes in yarn-cluster mode. I'm hitting this issue with
yarn-client mode only.
{code}
spark-submit --class org.apache.spark.examples.mllib.DecisionTreeRunner
--master yarn-client --jars hadoop-lzo-*.jar /xxx/lib/spark-examples_*jar
/tmp/sparkMLLInput/sample_libsvm_data.txt{code}
{code}
16/05/27 02:37:50 INFO SparkContext: Starting job: countByValue at
DecisionTreeRunner.scala:185
Exception in thread "dag-scheduler-event-loop" java.lang.NoClassDefFoundError:
org/apache/avro/generic/GenericRecord
at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:112)
at
org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:274)
at
org.apache.spark.serializer.KryoSerializerInstance.(KryoSerializer.scala:259)
at
org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:175)
at
org.apache.spark.serializer.KryoSerializer.supportsRelocationOfSerializedObjects$lzycompute(KryoSerializer.scala:182)
at
org.apache.spark.serializer.KryoSerializer.supportsRelocationOfSerializedObjects(KryoSerializer.scala:178)
at
org.apache.spark.shuffle.sort.SortShuffleManager$.canUseSerializedShuffle(SortShuffleManager.scala:187)
at
org.apache.spark.shuffle.sort.SortShuffleManager.registerShuffle(SortShuffleManager.scala:99)
at org.apache.spark.ShuffleDependency.(Dependency.scala:90)
at org.apache.spark.rdd.ShuffledRDD.getDependencies(ShuffledRDD.scala:91)
at org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:234)
at org.apache.spark.rdd.RDD$$anonfun$dependencies$2.apply(RDD.scala:232)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.rdd.RDD.dependencies(RDD.scala:232)
at org.apache.spark.scheduler.DAGScheduler.visit$1(DAGScheduler.scala:391)
at
org.apache.spark.scheduler.DAGScheduler.getParentStages(DAGScheduler.scala:403)
at
org.apache.spark.scheduler.DAGScheduler.getParentStagesAndId(DAGScheduler.scala:304)
at
org.apache.spark.scheduler.DAGScheduler.newResultStage(DAGScheduler.scala:339)
at
org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:849)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1626)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1618)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1607)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
Caused by: java.lang.ClassNotFoundException:
org.apache.avro.generic.GenericRecord
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
.. 23 more{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-15830) Spark application should get hive tokens only when it is required
[
https://issues.apache.org/jira/browse/SPARK-15830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora updated SPARK-15830:
---
Summary: Spark application should get hive tokens only when it is required
(was: Spark application should get hive tokens only when needed)
> Spark application should get hive tokens only when it is required
> -
>
> Key: SPARK-15830
> URL: https://issues.apache.org/jira/browse/SPARK-15830
> Project: Spark
> Issue Type: Improvement
>Reporter: Yesha Vora
>
> Currently , All spark application try to get Hive tokens (Even if application
> does not use them) if Hive is installed on the cluster.
> Due to this practice, spark application which does not require Hive fails
> when Hive service (metastore) is down for some reason.
> Thus, spark should only try to get Hive tokens when required. It should not
> fetch hive token if it is not needed by application.
> Example : Spark Pi application does not perform any hive related actions. But
> Spark Pi application still fails if hive metastore service is down.
> {code}
> 16/06/08 01:18:42 INFO YarnSparkHadoopUtil: getting token for namenode:
> hdfs://xxx:8020/user/xx/.sparkStaging/application_1465347287950_0001
> 16/06/08 01:18:42 INFO DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for
> xx on xx.xx.xx.xxx:8020
> 16/06/08 01:18:43 INFO metastore: Trying to connect to metastore with URI
> thrift://xx.xx.xx.xxx:9090
> 16/06/08 01:18:43 WARN metastore: Failed to connect to the MetaStore Server...
> 16/06/08 01:18:43 INFO metastore: Waiting 5 seconds before next connection
> attempt.
> 16/06/08 01:18:48 INFO metastore: Trying to connect to metastore with URI
> thrift://xx.xx.xx.xxx:9090
> 16/06/08 01:18:48 WARN metastore: Failed to connect to the MetaStore Server...
> 16/06/08 01:18:48 INFO metastore: Waiting 5 seconds before next connection
> attempt.
> 16/06/08 01:18:53 INFO metastore: Trying to connect to metastore with URI
> thrift://xx.xx.xx.xxx:9090
> 16/06/08 01:18:53 WARN metastore: Failed to connect to the MetaStore Server...
> 16/06/08 01:18:53 INFO metastore: Waiting 5 seconds before next connection
> attempt.
> 16/06/08 01:18:59 WARN Hive: Failed to access metastore. This class should
> not accessed in runtime.
> org.apache.hadoop.hive.ql.metadata.Hive Exception : java.lang.Runtime
> Exception : Unable to instantiate
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
> at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1236)
> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
> at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:166)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-15830) Spark application should get hive tokens only when needed
Yesha Vora created SPARK-15830:
--
Summary: Spark application should get hive tokens only when needed
Key: SPARK-15830
URL: https://issues.apache.org/jira/browse/SPARK-15830
Project: Spark
Issue Type: Improvement
Reporter: Yesha Vora
Currently , All spark application try to get Hive tokens (Even if application
does not use them) if Hive is installed on the cluster.
Due to this practice, spark application which does not require Hive fails when
Hive service (metastore) is down for some reason.
Thus, spark should only try to get Hive tokens when required. It should not
fetch hive token if it is not needed by application.
Example : Spark Pi application does not perform any hive related actions. But
Spark Pi application still fails if hive metastore service is down.
{code}
16/06/08 01:18:42 INFO YarnSparkHadoopUtil: getting token for namenode:
hdfs://xxx:8020/user/xx/.sparkStaging/application_1465347287950_0001
16/06/08 01:18:42 INFO DFSClient: Created HDFS_DELEGATION_TOKEN token 7 for xx
on xx.xx.xx.xxx:8020
16/06/08 01:18:43 INFO metastore: Trying to connect to metastore with URI
thrift://xx.xx.xx.xxx:9090
16/06/08 01:18:43 WARN metastore: Failed to connect to the MetaStore Server...
16/06/08 01:18:43 INFO metastore: Waiting 5 seconds before next connection
attempt.
16/06/08 01:18:48 INFO metastore: Trying to connect to metastore with URI
thrift://xx.xx.xx.xxx:9090
16/06/08 01:18:48 WARN metastore: Failed to connect to the MetaStore Server...
16/06/08 01:18:48 INFO metastore: Waiting 5 seconds before next connection
attempt.
16/06/08 01:18:53 INFO metastore: Trying to connect to metastore with URI
thrift://xx.xx.xx.xxx:9090
16/06/08 01:18:53 WARN metastore: Failed to connect to the MetaStore Server...
16/06/08 01:18:53 INFO metastore: Waiting 5 seconds before next connection
attempt.
16/06/08 01:18:59 WARN Hive: Failed to access metastore. This class should not
accessed in runtime.
org.apache.hadoop.hive.ql.metadata.Hive Exception : java.lang.Runtime Exception
: Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1236)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:166)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498){code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-15283) YARN Application should get FAILED state if driver is failed in yarn-client mode
Yesha Vora created SPARK-15283: -- Summary: YARN Application should get FAILED state if driver is failed in yarn-client mode Key: SPARK-15283 URL: https://issues.apache.org/jira/browse/SPARK-15283 Project: Spark Issue Type: Bug Reporter: Yesha Vora If a spark application is running in yarn-client mode and if its driver fails, the yarn application status remains SUCCEEDED. If spark application stops/ fails due to driver failure, the yarn application status should be FAILED. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-14466) spark_classpath can not handle regex if it is specified in spark-env.sh
[
https://issues.apache.org/jira/browse/SPARK-14466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yesha Vora resolved SPARK-14466.
Resolution: Invalid
Thanks [~srowen]. Closing this jira as Invalid
> spark_classpath can not handle regex if it is specified in spark-env.sh
> ---
>
> Key: SPARK-14466
> URL: https://issues.apache.org/jira/browse/SPARK-14466
> Project: Spark
> Issue Type: Bug
>Reporter: Yesha Vora
>
> If spark_classpath is set in spark-env.sh, It can not handle regex(\*)
> properly.
> {code}
> export
> SPARK_CLASSPATH=/xyz/lib/hbase-common.jar:/xyz/lib/hbase-client.jar:/xyz/lib/hbase-server.jar:/xyz/lib/hbase-protocol.jar:/xyz/lib/guava-*.jar
> {code}
> Here, spark can not find guava-12.0.1.jar . It does not handle \* regex
> properly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-14466) spark_classpath can not handle regex if it is specified in spark-env.sh
Yesha Vora created SPARK-14466:
--
Summary: spark_classpath can not handle regex if it is specified
in spark-env.sh
Key: SPARK-14466
URL: https://issues.apache.org/jira/browse/SPARK-14466
Project: Spark
Issue Type: Bug
Reporter: Yesha Vora
If spark_classpath is set in spark-env.sh, It can not handle regex(\*) properly.
{code}
export
SPARK_CLASSPATH=/xyz/lib/hbase-common.jar:/xyz/lib/hbase-client.jar:/xyz/lib/hbase-server.jar:/xyz/lib/hbase-protocol.jar:/xyz/lib/guava-*.jar
{code}
Here, spark can not find guava-12.0.1.jar . It does not handle \* regex
properly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-8929) [Windows] Application with Appname including whiteSpace fails in Yarn-client mode
Yesha Vora created SPARK-8929:
-
Summary: [Windows] Application with Appname including whiteSpace
fails in Yarn-client mode
Key: SPARK-8929
URL: https://issues.apache.org/jira/browse/SPARK-8929
Project: Spark
Issue Type: Bug
Reporter: Yesha Vora
Some MachineLearning examples has space in App Name. These applications fail to
start AM with yarn-client mode in Windows environment.
Affected Test example list:
* BinaryClassification
* Correlations
* DecisionTreeRunner
* DenseKmeans
* GradientBoostedTreesRunner
* LinearRegression
* MovieLensALS
* MultivariateSummarizer
* SampledRDDs
* SparseNaiveBayes
{code:title=SampledRDDs}
RUNNING: call spark-submit --class org.apache.spark.examples.mllib.SampledRDDs
--master yarn-client --properties-file c:\windows\temp\spark-defaults.conf
--jars spark-examples-*.jar --input
/tmp/sparkMLLInput/sample_binary_classification_data.txt
{code}
{code:title=Fails to Start AM}
2015-06-13 22:07:27,526|beaver.machine|INFO||7600|MainThread|Container id:
container_e02_1434177640451_0029_02_01
2015-06-13 22:07:27,526|beaver.machine|INFO||7600|MainThread|Exit code: 9009
2015-06-13 22:07:27,528|beaver.machine|INFO||7600|MainThread|Exception
message: Usage: java [-options] class [args...]
2015-06-13 22:07:27,529|beaver.machine|INFO||7600|MainThread|(to execute a
class)
2015-06-13 22:07:27,529|beaver.machine|INFO||7600|MainThread|or java
[-options] -jar jarfile [args...]
2015-06-13 22:07:27,529|beaver.machine|INFO||7600|MainThread|(to execute a
jar file)
2015-06-13 22:07:27,529|beaver.machine|INFO||7600|MainThread|where options
include:
2015-06-13 22:07:27,530|beaver.machine|INFO||7600|MainThread|-d32 use a
32-bit data model if available
2015-06-13 22:07:27,530|beaver.machine|INFO||7600|MainThread|-d64 use a
64-bit data model if available
2015-06-13 22:07:27,532|beaver.machine|INFO||7600|MainThread|-server
to select the "server" VM
2015-06-13 22:07:27,532|beaver.machine|INFO||7600|MainThread|The default VM
is server.
2015-06-13 22:07:27,532|beaver.machine|INFO||7600|MainThread|
2015-06-13 22:07:27,532|beaver.machine|INFO||7600|MainThread|-cp
2015-06-13 22:07:27,533|beaver.machine|INFO||7600|MainThread|-classpath
2015-06-13 22:07:27,533|beaver.machine|INFO||7600|MainThread|A ; separated
list of directories, JAR archives,
2015-06-13 22:07:27,535|beaver.machine|INFO||7600|MainThread|and ZIP
archives to search for class files.
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
