TengHuo commented on PR #5391:
URL: https://github.com/apache/hudi/pull/5391#issuecomment-1362587097
And this is an simple example to produce this error `exitCode: 16` for
proving `System.exit` is not appropriate way
```scala
object SparkExitTest {
val spark: SparkSession = SparkSession
.builder
.appName("Test_HuoTeng_Spark_Exit")
.config("spark.driver.memory", "6g")
.config("spark.executor.memory", "4g")
.config("spark.executor.instances", 1)
.config("spark.executor.cores", 1)
.config("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
.getOrCreate()
def main(args: Array[String]): Unit = {
var ret = 0
try {
ret = 1
} catch {
case throwable: Throwable =>
println("Fail to run because " + throwable)
} finally {
spark.close()
System.exit(ret)
}
}
}
```
Submit this Spark application in Yarn cluster mode will get this error
```log
22/12/22 17:05:36 INFO SparkContext: Successfully stopped SparkContext
22/12/22 17:05:36 INFO ApplicationMaster: Final app status: FAILED,
exitCode: 16, (reason: Shutdown hook called before final status was reported.)
22/12/22 17:05:36 INFO ApplicationMaster: Unregistering ApplicationMaster
with FAILED (diag message: Shutdown hook called before final status was
reported.)
22/12/22 17:05:36 INFO AMRMClientImpl: Waiting for application to be
successfully unregistered.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
