[
https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hustfxj updated SPARK-19264:
----------------------------
Description:
I think work can't start driver by "ProcessBuilderLike", thus we can't know
the application's main thread is finished or not if the application's main
thread contains some non-daemon threads. Because the program terminates when
there no longer is any non-daemon thread running (or someone called
System.exit). The main thread can have finished long ago.
worker should start driver like AM of YARN . As followed:
{code:title=ApplicationMaster.scala|borderStyle=solid}
mainMethod.invoke(null, userArgs.toArray)
finish(FinalApplicationStatus.SUCCEEDED, ApplicationMaster.EXIT_SUCCESS)
logDebug("Done running users class")
{code}
Then the work can monitor the driver's main thread, and know the application's
state.
was:
I think work can't start driver by "ProcessBuilderLike", thus we can't know
the application's main thread is finished or not if the application's main
thread contains some daemon threads. Because the program terminates when there
no longer is any non-daemon thread running (or someone called System.exit). The
main thread can have finished long ago.
worker should start driver like AM of YARN . As followed:
{code:title=ApplicationMaster.scala|borderStyle=solid}
mainMethod.invoke(null, userArgs.toArray)
finish(FinalApplicationStatus.SUCCEEDED, ApplicationMaster.EXIT_SUCCESS)
logDebug("Done running users class")
{code}
Then the work can monitor the driver's main thread, and know the application's
state.
> Work should start driver, the same to AM of yarn
> ---------------------------------------------------
>
> Key: SPARK-19264
> URL: https://issues.apache.org/jira/browse/SPARK-19264
> Project: Spark
> Issue Type: Improvement
> Reporter: hustfxj
>
> I think work can't start driver by "ProcessBuilderLike", thus we can't
> know the application's main thread is finished or not if the application's
> main thread contains some non-daemon threads. Because the program terminates
> when there no longer is any non-daemon thread running (or someone called
> System.exit). The main thread can have finished long ago.
> worker should start driver like AM of YARN . As followed:
> {code:title=ApplicationMaster.scala|borderStyle=solid}
> mainMethod.invoke(null, userArgs.toArray)
> finish(FinalApplicationStatus.SUCCEEDED, ApplicationMaster.EXIT_SUCCESS)
> logDebug("Done running users class")
> {code}
> Then the work can monitor the driver's main thread, and know the
> application's state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
