Jave-Chen commented on a change in pull request #2224: support custom datax
configuration
URL:
https://github.com/apache/incubator-dolphinscheduler/pull/2224#discussion_r394733232
##########
File path:
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/datax/DataxTask.java
##########
@@ -192,24 +192,47 @@ private String buildDataxJsonFile()
throws Exception {
// generate json
String fileName = String.format("%s/%s_job.json", taskDir,
taskProps.getTaskAppId());
+ String json;
Path path = new File(fileName).toPath();
if (Files.exists(path)) {
return fileName;
}
- JSONObject job = new JSONObject();
- job.put("content", buildDataxJobContentJson());
- job.put("setting", buildDataxJobSettingJson());
- JSONObject root = new JSONObject();
- root.put("job", job);
- root.put("core", buildDataxCoreJson());
- logger.debug("datax job json : {}", root.toString());
+ if (dataXParameters.getCustomConfig() == 1){
+
+ json = dataXParameters.getJson().replaceAll("\\r\\n", "\n");
+
Review comment:
Variables can also be used in datax job json, which will cause json to be
invalid, such as
```
{
...
"reader": {
"name": "mysqlreader",
"parameter": {
"username": "${db_user}",
"password": "${db_pwd}",
"encoding": "UTF-8",
"column": [${tbl_clms}],
"connection": [
{
"jdbcUrl": [
"jdbc:mysql://${host}/${db}?useUnicode=true&characterEncoding=utf8"
],
"table": [
${table_list}
]
}
]
}
}
...
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
