wen-hemin commented on issue #5241:
URL:
https://github.com/apache/incubator-dolphinscheduler/issues/5241#issuecomment-818640603
> Yeal, this problem does happen in the dev branch. This may be due to the
job.json file created by `dolphinscheduler` can't deserialize by `datax`, but I
am not sure if it's because of the version.
>
> The job.json created by ds is look like below:
>
> ```json
>
{"job":{"content":[{"reader":"{\"name\":\"clickhousereader\",\"parameter\":{\"username\":\"default\",\"password\":\"123456\",\"connection\":[{\"querySql\":[\"select
user_name from
ds_user\"],\"jdbcUrl\":[\"jdbc:clickhouse://localhost:8123/default\"]}]}}","writer":"{\"name\":\"mysqlwriter\",\"parameter\":{\"username\":\"root\",\"password\":\"123456\",\"column\":[\"`user_name`\"],\"connection\":[{\"table\":[\"t_ds_user\"],\"jdbcUrl\":\"jdbc:mysql://localhost:3308/dolphinscheduler?allowLoadLocalInfile=false&autoDeserialize=false&allowLocalInfile=false&allowUrlInLocalInfile=false\"}]}}"}],"setting":{"speed":"{\"channel\":1,\"record\":1000}","errorLimit":"{\"record\":0,\"percentage\":0}"}},"core":{"transport":{"channel":{"speed":{"channel":1,"record":1000}}}}}
> ```
>
> format
>
> ```json
> {
> "job":{
> "content":[
> {
>
"reader":"{"name":"clickhousereader","parameter":{"username":"default","password":"123456","connection":[{"querySql":["select
user_name from
ds_user"],"jdbcUrl":["jdbc:clickhouse://localhost:8123/default"]}]}}",
>
"writer":"{"name":"mysqlwriter","parameter":{"username":"root","password":"123456","column":["`user_name`"],"connection":[{"table":["t_ds_user"],"jdbcUrl":"jdbc:mysql://localhost:3308/dolphinscheduler?allowLoadLocalInfile=false&autoDeserialize=false&allowLocalInfile=false&allowUrlInLocalInfile=false"}]}}"
> }
> ],
> "setting":{
> "speed":"{"channel":1,"record":1000}",
> "errorLimit":"{"record":0,"percentage":0}"
> }
> },
> "core":{
> "transport":{
> "channel":{
> "speed":{
> "channel":1,
> "record":1000
> }
> }
> }
> }
> }
> ```
>
> The value of reader、writer and setting is a string, this may not be parsed
by `datax`. So we may get error such as `ConfigParser-plugin load failed` etc.
> The simple way to solve is remove toString() in line 313、314、344、345 like
`content.put("reader", reader);` Furthermore, I suggest to use a class DataXJob
to store the config of datax job setting instead of using ObjectNode.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
