Andy-xu-007 commented on issue #98:
URL:
https://github.com/apache/dolphinscheduler-sdk-python/issues/98#issuecomment-1752275308
YES, I have the same problem while using the pydolphinscheduler CLI to
create workflow with YAML file, pydolphinscheduler's version im using is 4.0.3,
and the version of dolphinscheduler is 3.1.7, a part of whole code as following:
`
- name: child2
task_type: Shell
command: sh test1.sh
fail_retry_times: 2
fail_retry_interval: 3
worker_group: worker252
environment_name: worker-[252]-environment
resource_list: [test1.sh]
deps:
- parent
`
I'm used these code to generate workflow and the programe raise an error
like below:
`
Auth token is default token, highly recommend add a token in production,
especially you deploy in public network.
/home/hadoop/.local/lib/python3.8/site-packages/pydolphinscheduler/java_gateway.py:324:
UserWarning: Using unmatched version of pydolphinscheduler (version 4.0.3) and
Java gateway (version 3.1.7) may cause errors. We strongly recommend you to
find the matched version (check:
https://pypi.org/project/apache-dolphinscheduler)
gateway = GatewayEntryPoint()
Traceback (most recent call last):
File "/home/hadoop/.local/bin/pydolphinscheduler", line 8, in <module>
sys.exit(cli())
File "/home/hadoop/.local/lib/python3.8/site-packages/click/core.py", line
1157, in __call__
return self.main(*args, **kwargs)
File "/home/hadoop/.local/lib/python3.8/site-packages/click/core.py", line
1078, in main
rv = self.invoke(ctx)
File "/home/hadoop/.local/lib/python3.8/site-packages/click/core.py", line
1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/hadoop/.local/lib/python3.8/site-packages/click/core.py", line
1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/hadoop/.local/lib/python3.8/site-packages/click/core.py", line
783, in invoke
return __callback(*args, **kwargs)
File
"/home/hadoop/.local/lib/python3.8/site-packages/pydolphinscheduler/cli/commands.py",
line 106, in yaml
create_workflow(yaml_file)
File
"/home/hadoop/.local/lib/python3.8/site-packages/pydolphinscheduler/core/yaml_workflow.py",
line 494, in create_workflow
YamlWorkflow.parse(yaml_file)
File
"/home/hadoop/.local/lib/python3.8/site-packages/pydolphinscheduler/core/yaml_workflow.py",
line 235, in parse
workflow_name = cls(yaml_file).create_workflow()
File
"/home/hadoop/.local/lib/python3.8/site-packages/pydolphinscheduler/core/yaml_workflow.py",
line 179, in create_workflow
task = self.parse_task(task_data, name2task)
File
"/home/hadoop/.local/lib/python3.8/site-packages/pydolphinscheduler/core/yaml_workflow.py",
line 290, in parse_task
task = task_cls(**task_params)
File
"/home/hadoop/.local/lib/python3.8/site-packages/pydolphinscheduler/tasks/shell.py",
line 58, in __init__
super().__init__(name, TaskType.SHELL, *args, **kwargs)
File
"/home/hadoop/.local/lib/python3.8/site-packages/pydolphinscheduler/core/task.py",
line 222, in __init__
self.get_content()
File
"/home/hadoop/.local/lib/python3.8/site-packages/pydolphinscheduler/core/task.py",
line 332, in get_content
res = self.get_plugin()
File
"/home/hadoop/.local/lib/python3.8/site-packages/pydolphinscheduler/core/task.py",
line 319, in get_plugin
raise PyResPluginException(
pydolphinscheduler.exceptions.PyResPluginException: The execution command of
this task is a file, but the resource plugin is empty
`
the error remaind me to add resource plugin in YMAL file, resource center
build on HDFS, after checking the source code and official doc,
pydolphinscheduler support Local, GitHub, GitLab, OSS, S3, but not contain
HDFS, so i should create a new resource plugin with HDFS? or do you have some
other way can help me to create a workflow which can using resource center file
with YAML file
[Looking forward to your reply](javascript:;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
