Radeity opened a new pull request, #12197: URL: https://github.com/apache/dolphinscheduler/pull/12197
<!--Thanks very much for contributing to Apache DolphinScheduler. Please review https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html before opening a pull request.--> ## Purpose of the pull request - Import `aop` way as an alternative way to collect yarn job's applicationId. - Also, this PR closes https://github.com/apache/dolphinscheduler/issues/11262 ## Brief change log - Create new module `dolphinscheduler-aop` for `aop` code. - Add new environment configuration for each type of yarn tasks to support `aop`, as follow: ```shell export HADOOP_CLASSPATH=${DOLPHINSCHEDULER_HOME}/tools/libs/aop/*:$HADOOP_CLASSPATH export SPARK_DIST_CLASSPATH=${DOLPHINSCHEDULER_HOME}/tools/libs/aop/*:$SPARK_DIST_CLASS_PATH export HADOOP_CLIENT_OPTS="-javaagent:${DOLPHINSCHEDULER_HOME}/tools/libs/aspectjweaver-1.9.7.jar":$HADOOP_CLIENT_OPTS export SPARK_SUBMIT_OPTS="-javaagent:${DOLPHINSCHEDULER_HOME}/tools/libs/aspectjweaver-1.9.7.jar":$SPARK_SUBMIT_OPTS export FLINK_ENV_JAVA_OPTS="-javaagent:${DOLPHINSCHEDULER_HOME}/tools/libs/aspectjweaver-1.9.7.jar":$FLINK_ENV_JAVA_OPTS ``` - Add user properties for user to decide how to collect applicationId, as follow: ```shell # way to collect applicationId: log(original regex match), aop appId.collect: log # appIds info log path appId.file.path = appInfo.log ``` ## Verify this pull request - Service logic is verified by unit test. - Effectiveness of `aop` for supported types of yarn job are simply tested on my local cluster. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
