[ https://issues.apache.org/jira/browse/GRIFFIN-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16664963#comment-16664963 ]
ASF GitHub Bot commented on GRIFFIN-208: ---------------------------------------- Github user bhlx3lyx7 commented on a diff in the pull request: https://github.com/apache/incubator-griffin/pull/448#discussion_r228469933 --- Diff: measure/src/main/scala/org/apache/griffin/measure/Application.scala --- @@ -104,12 +106,16 @@ object Application extends Loggable { case Success(_) => info("process end success") case Failure(ex) => - error(s"process end error: ${ex.getMessage}") + error(s"process end error: ${ex.getMessage}", ex) shutdown sys.exit(-5) } shutdown + + if (!success) { + sys.exit(-5) + } --- End diff -- If any rule step fails, explicitly break down the application can let the spark application shut down with a FAILED state. It makes sense. Actually, we can think about how to get the job state. Through livy or yarn, we can only get the lifecycle state of spark job, we still need to check logs to address the detailed error, and it would be much more difficult for common users. We can define some job states in calculation, like `starting`, `loading data source`, `pre-processing`, `rule step N success`, and let spark job report its state by configured notify method like http request to service side or something else. In such a way, service can manage states of job instances. I think job state management is the final solution, explicitly exit the application could work as a temporary solution at current. > Job status is SUCCESS even if some stages have failed > ----------------------------------------------------- > > Key: GRIFFIN-208 > URL: https://issues.apache.org/jira/browse/GRIFFIN-208 > Project: Griffin (Incubating) > Issue Type: Bug > Reporter: Nikolay Sokolov > Priority: Major > > When some steps (MetricWrite or SparkSql, for example) fail, errors are just > logged, but not reported as part of job status. Symptoms: > {code:none} > 18/10/22 17:17:58 ERROR transform.SparkSqlTransformStep: run spark sql [ .... > ] error: ... > {code} > YarnApplicationState: FINISHED > FinalStatus Reported by AM: SUCCEEDED -- This message was sent by Atlassian JIRA (v7.6.3#76005)