chengshiwen edited a comment on issue #5648: URL: https://github.com/apache/dolphinscheduler/issues/5648#issuecomment-868479357
@geosmart @blackberrier I think the following things should be considered: 1. extend `SparkArgsUtils.java` and `spark.vue` 2. adapt to geting the spark job state using `namespace` and `<driver-pod-name>`, instead of calling `isSuccessOfYarnState` and `yarn rest api` 3. adapt to kill spark job using `namespace` and `<driver-pod-name>`, instead of `yarn application -kill` 4. check the spark application name that is specified by `spark.app.name` or the `--name` argument whch must consist of lower case alphanumeric characters, -, and . and must start and end with an alphanumeric character Things that should not be done by us: 1. change dolphinscheudler from hdfs to minio (dependencies hosted in remote locations like HDFS or HTTP servers should be solved by user, like using pvc to pre-mount them) 2. yarn log aggregation (third-party framework should solve it) 3. build spark docker images (not our duty, user will customize the image to match an individual application’s needs) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
