GitHub user guoxiaolongzte opened a pull request:
https://github.com/apache/spark/pull/17696
[SPARK-20401][DOC]In the spark official configuration document, the
'spark.driver.supervise' configuration parameter specification and default
values are necessary.
## What changes were proposed in this pull request?
Use the REST interface submits the spark job.
e.g.
curl -X POST http://10.43.183.120:6066/v1/submissions/create --header
"Content-Type:application/json;charset=UTF-8" --data'{
"action": "CreateSubmissionRequest",
"appArgs": [
"myAppArgument"
],
"appResource": "/home/mr/gxl/test.jar",
"clientSparkVersion": "2.2.0",
"environmentVariables": {
"SPARK_ENV_LOADED": "1"
},
"mainClass": "cn.zte.HdfsTest",
"sparkProperties": {
"spark.jars": "/home/mr/gxl/test.jar",
**"spark.driver.supervise": "true",**
"spark.app.name": "HdfsTest",
"spark.eventLog.enabled": "false",
"spark.submit.deployMode": "cluster",
"spark.master": "spark://10.43.183.120:6066"
}
}'
**I hope that make sure that the driver is automatically restarted if it
fails with non-zero exit code.
But I can not find the 'spark.driver.supervise' configuration parameter
specification and default values from the spark official document.**
## How was this patch tested?
manual tests
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/guoxiaolongzte/spark SPARK-20401
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17696.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17696
----
commit d383efba12c66addb17006dea107bb0421d50bc3
Author: éå°é¾ 10207633 <[email protected]>
Date: 2017-03-31T13:57:09Z
[SPARK-20177]Document about compression way has some little detail changes.
commit 3059013e9d2aec76def14eb314b6761bea0e7ca0
Author: éå°é¾ 10207633 <[email protected]>
Date: 2017-04-01T01:38:02Z
[SPARK-20177] event log add a space
commit 555cef88fe09134ac98fd0ad056121c7df2539aa
Author: guoxiaolongzte <[email protected]>
Date: 2017-04-02T00:16:08Z
'/applications/[app-id]/jobs' in rest api,status should be
[running|succeeded|failed|unknown]
commit 46bb1ad3ddd9fb55b5607ac4f20213a90186cfe9
Author: éå°é¾ 10207633 <[email protected]>
Date: 2017-04-05T03:16:50Z
Merge branch 'master' of https://github.com/apache/spark into SPARK-20177
commit 0efb0dd9e404229cce638fe3fb0c966276784df7
Author: éå°é¾ 10207633 <[email protected]>
Date: 2017-04-05T03:47:53Z
[SPARK-20218]'/applications/[app-id]/stages' in REST API,add description.
commit 0e37fdeee28e31fc97436dabd001d3c85c5a7794
Author: éå°é¾ 10207633 <[email protected]>
Date: 2017-04-05T05:22:54Z
[SPARK-20218] '/applications/[app-id]/stages/[stage-id]' in REST API,remove
redundant description.
commit 52641bb01e55b48bd9e8579fea217439d14c7dc7
Author: éå°é¾ 10207633 <[email protected]>
Date: 2017-04-07T06:24:58Z
Merge branch 'SPARK-20218'
commit d3977c9cab0722d279e3fae7aacbd4eb944c22f6
Author: éå°é¾ 10207633 <[email protected]>
Date: 2017-04-08T07:13:02Z
Merge branch 'master' of https://github.com/apache/spark
commit 137b90e5a85cde7e9b904b3e5ea0bb52518c4716
Author: éå°é¾ 10207633 <[email protected]>
Date: 2017-04-10T05:13:40Z
Merge branch 'master' of https://github.com/apache/spark
commit 0fe5865b8022aeacdb2d194699b990d8467f7a0a
Author: éå°é¾ 10207633 <[email protected]>
Date: 2017-04-10T10:25:22Z
Merge branch 'SPARK-20190' of https://github.com/guoxiaolongzte/spark
commit cf6f42ac84466960f2232c025b8faeb5d7378fe1
Author: éå°é¾ 10207633 <[email protected]>
Date: 2017-04-10T10:26:27Z
Merge branch 'master' of https://github.com/apache/spark
commit 685cd6b6e3799c7be65674b2670159ba725f0b8f
Author: éå°é¾ 10207633 <[email protected]>
Date: 2017-04-14T01:12:41Z
Merge branch 'master' of https://github.com/apache/spark
commit c716a9231e9ab117d2b03ba67a1c8903d8d9da93
Author: guoxiaolong <[email protected]>
Date: 2017-04-17T06:57:21Z
Merge branch 'master' of https://github.com/apache/spark
commit 679cec36a968fbf995b567ca5f6f8cbd8e32673f
Author: guoxiaolong <[email protected]>
Date: 2017-04-19T07:20:08Z
Merge branch 'master' of https://github.com/apache/spark
commit 3c9387af84a8f39cf8c1ce19e15de99dfcaf0ca5
Author: guoxiaolong <[email protected]>
Date: 2017-04-19T08:15:26Z
Merge branch 'master' of https://github.com/apache/spark
commit cb71f4462a0889cbb0843875b1e4cf14bcb0d020
Author: guoxiaolong <[email protected]>
Date: 2017-04-20T05:52:06Z
Merge branch 'master' of https://github.com/apache/spark
commit 3e74d731a34c2ee112577a4118b6cbf706227120
Author: guoxiaolong <[email protected]>
Date: 2017-04-20T06:26:51Z
[SPARK-20401]In the spark official configuration document, the
'spark.driver.supervise' configuration parameter specification and default
values are necessary.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
