非常开心收到dolphinscheduler 开发者邮件,这次提问主要是因为上传文件问题,dolphinscheduler
大部分已经满足实际需求,但也有使用时候遇到些问题,目前,主要发现两个问题:
1、上传文件只能是文件而不是文件夹;
2、上传资源只能单独授权某一个用户,维护成本高;
在 2021-06-05 23:21:30,"Lidong Dai" <[email protected]> 写道:
>hi,
> any progress? Do we need a meeting to solve this?
>By the way, the picture can't show in the apache mailing list, you can
>upload the pic to github, then paste the url address to the mail.
>
>
>Best Regards
>---------------
>DolphinScheduler PMC
>Lidong Dai
>[email protected]
>---------------
>
>
>On Tue, Jun 1, 2021 at 5:04 PM Xingjie Wang(联通集团联通数字科技有限公司) <
>[email protected]> wrote:
>
>> This really doesn't satisfy those scene.
>> For the first one ,If we want to do this, we should save the varPool to
>> the level of processInstance, so the Task4 can get the varPool from Task1
>> and do not by Task2 and Task3.
>> This one will obscure the relation between globalParam and localParam and
>> varPool.
>> Other plan ,when user define the Task4 IN param ,user should chonse the
>> Task1 that this param is the Task1's OUT param .When init the Task4's
>> varPool ,get the Task1 form completeTaskList,then get the varPool.And when
>> the taskInstance have a new property, mark this taskInstance'name into this
>> property ,and put this property into the varPool.
>> It will satisfy those scene.
>> How do you think?
>> -----邮件原件-----
>> 发件人: Ruan, Wenjun <[email protected]>
>> 发送时间: 2021年6月1日 16:07
>> 收件人: [email protected]
>> 主题: Re: [DISCUSS]The new Plan of global params
>>
>> Sorry, it seems that the picture cannot be displayed well, the dag
>> structure is as follow:
>>
>> Task1 -> Task2 -> Task3 -> Task4
>>
>> From: Ruan, Wenjun <[email protected]>
>> Date: Tuesday, June 1, 2021 at 3:55 PM
>> To: [email protected] <[email protected]>
>> Subject: Re: [DISCUSS]The new Plan of global params External Email Hi
>> Xingjie,
>>
>> I have two things want to confirm.
>> In your plan it seems that we need to store varpool in all the post nodes?
>> For example, if I have a simple dag like below:
>> [cid:[email protected]]
>>
>> If I want to get the out param of Task1 in Task4, for example we call it
>> parameterA, then we need to store parameterA in Task2’s varpool and Task3’s
>> varpool, even if we don’t need parameterA in Task2 and Task3, am I right?
>> Can we take it directly from Task1?
>>
>> The second thing is that if I want to troubleshouting, can I find the
>> source of the parameters in the varpool? It seems I need to look forward
>> and find the first node that contains the parameter? It seems not
>> convenient, if I have much tasks in a dag.
>>
>> Thanks,
>> Wenjun Ruan
>>
>>
>> From: Xingjie Wang(联通集团联通数字科技有限公司) <[email protected]>
>> Date: Tuesday, June 1, 2021 at 3:21 PM
>> To: [email protected] <[email protected]>
>> Subject: [DISCUSS]The new Plan of global params External Email
>>
>>
>>
>> Hi Dev Team
>>
>> The scheme of The global params that Task need change as blow。
>> Here is the ISSUE about this DISCUSS。
>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fdolphinscheduler%2Fissues%2F5565&data=04%7C01%7Cweruan%40ebay.com%7C71c6b15115994fd6463808d924cdbf7b%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637581288947116284%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Na6C8tAQODl%2BrfThuaGmtCcVqqwt1eSBpQTFgaRuVXc%3D&reserved=0
>> <
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fdolphinscheduler%2Fissues%2F5565&data=04%7C01%7Cweruan%40ebay.com%7C2a73397f8b394100b21508d924d28b37%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637581309371279320%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=1B%2BKIBz%2BCfhTp0nRiK3oKFz4KKgYQDN%2B46xF091o0BQ%3D&reserved=0
>> >
>>
>> I will change the scheme of global param that depend on the relation of
>> the task.
>> Here is the detail about this .
>>
>> 1. create taskInstance
>>
>> Get the previous tasks ,get the varPool of those tasks, put those into
>> this varPool. If the previous task has the same varPool name,use the value
>> that is not null.If all of the values are null, use the earlier one.
>>
>> 1. Worker get the param
>>
>> Master will send the varPool to the Worker ,and taskPorcessor get the
>> varPool with the format of List<Property>, varPool will do the same as the
>> localParam.
>>
>> 1. Worker response out Parm
>>
>> When the user define the OUT param in the page of Task Definition.Worker
>> will get the result of Processor .
>>
>> The different Processor return the different format ,for example SQL
>> return the format of List<Map<String,String>>,users could get more than one
>> line or more than one column;SHELL get the Map<String,String> or
>> String.This out Param will add into the varPool ,and send to the Master
>> ,save into the databases.Also ,this value will save into the localParam
>>
>>
>>
>> If you have some question or have the better plan ,please contact me
>> .thank you.
>>
>> 如果您错误接收了该邮件,请通过电子邮件立即通知我们。请回复邮件到
>> [email protected],即可以退订此邮件。我们将立即将您的信息从我们的发送目录中删除。
>> If you have received this email in error please notify us immediately by
>> e-mail. Please reply to [email protected] ,you can unsubscribe from
>> this mail. We will immediately remove your information from send catalogue
>> of our.
>> 如果您错误接收了该邮件,请通过电子邮件立即通知我们。请回复邮件到
>> [email protected],即可以退订此邮件。我们将立即将您的信息从我们的发送目录中删除。
>> If you have received this email in error please notify us immediately by
>> e-mail. Please reply to [email protected] ,you can unsubscribe from
>> this mail. We will immediately remove your information from send catalogue
>> of our.
>>