zhongjiajie commented on issue #6407:
URL:
https://github.com/apache/dolphinscheduler/issues/6407#issuecomment-939654995
Hi @zuston, I answered this question in mailing-list you could see in this
link[1]
> But could we combine user-code(like flink-sql or spark program code) with
python workflow sdk? This maybe more convenient and easy-understanding.
Yeah, this is the first step we make, but we wish it could be more powerful.
We could support python functional task later, such as
```python
@task
def func_task():
print('This function could run in DolphinScheduler')
@task
def func_task_follow():
print('A task run after `func_task` success')
with process_definition('workflow_name') as pd:
func_task >> func_task_follow
```
Which mean that our python could be run in DolphinScheduler. But I'm not
pretty sure could we run pyflink or pyspark code like that cause this need
flink and spark environment, but we hope it could.
If you have idea please let me know, or you could join us to build this
feature together if you interested in this
[1]:
https://lists.apache.org/thread.html/ra4fd314479930ac91efdf6917b9747f3328c8378b6a1a50be2cc6465%40%3Cdev.dolphinscheduler.apache.org%3E
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
