hi, Glad to hear from what you said. Are you interested in doing some contributions to implement these features?
Best Regards --------------- DolphinScheduler PMC Lidong Dai [email protected] --------------- On Sun, Jun 6, 2021 at 3:28 PM 王峰 <[email protected]> wrote: > > > > 非常开心收到dolphinscheduler 开发者邮件，这次提问主要是因为上传文件问题，dolphinscheduler > 大部分已经满足实际需求，但也有使用时候遇到些问题，目前，主要发现两个问题： > 1、上传文件只能是文件而不是文件夹； > 2、上传资源只能单独授权某一个用户，维护成本高； > > > > > > > > > > > > > > > 在 2021-06-05 23:21:30，"Lidong Dai" <[email protected]> 写道： > >hi, > > any progress? Do we need a meeting to solve this? > >By the way, the picture can't show in the apache mailing list, you can > >upload the pic to github, then paste the url address to the mail. > > > > > >Best Regards > >--------------- > >DolphinScheduler PMC > >Lidong Dai > >[email protected] > >--------------- > > > > > >On Tue, Jun 1, 2021 at 5:04 PM Xingjie Wang(联通集团联通数字科技有限公司) < > >[email protected]> wrote: > > > >> This really doesn't satisfy those scene. > >> For the first one ,If we want to do this, we should save the varPool to > >> the level of processInstance, so the Task4 can get the varPool from > Task1 > >> and do not by Task2 and Task3. > >> This one will obscure the relation between globalParam and localParam > and > >> varPool. > >> Other plan ,when user define the Task4 IN param ,user should chonse the > >> Task1 that this param is the Task1's OUT param .When init the Task4's > >> varPool ,get the Task1 form completeTaskList,then get the varPool.And > when > >> the taskInstance have a new property, mark this taskInstance'name into > this > >> property ,and put this property into the varPool. > >> It will satisfy those scene. > >> How do you think? > >> -----邮件原件----- > >> 发件人: Ruan, Wenjun <[email protected]> > >> 发送时间: 2021年6月1日 16:07 > >> 收件人: [email protected] > >> 主题: Re: [DISCUSS]The new Plan of global params > >> > >> Sorry, it seems that the picture cannot be displayed well, the dag > >> structure is as follow: > >> > >> Task1 -> Task2 -> Task3 -> Task4 > >> > >> From: Ruan, Wenjun <[email protected]> > >> Date: Tuesday, June 1, 2021 at 3:55 PM > >> To: [email protected] <[email protected]> > >> Subject: Re: [DISCUSS]The new Plan of global params External Email Hi > >> Xingjie, > >> > >> I have two things want to confirm. > >> In your plan it seems that we need to store varpool in all the post > nodes? > >> For example, if I have a simple dag like below: > >> [cid:[email protected]] > >> > >> If I want to get the out param of Task1 in Task4, for example we call it > >> parameterA, then we need to store parameterA in Task2’s varpool and > Task3’s > >> varpool, even if we don’t need parameterA in Task2 and Task3, am I > right? > >> Can we take it directly from Task1? > >> > >> The second thing is that if I want to troubleshouting, can I find the > >> source of the parameters in the varpool? It seems I need to look forward > >> and find the first node that contains the parameter? It seems not > >> convenient, if I have much tasks in a dag. > >> > >> Thanks, > >> Wenjun Ruan > >> > >> > >> From: Xingjie Wang(联通集团联通数字科技有限公司) <[email protected]> > >> Date: Tuesday, June 1, 2021 at 3:21 PM > >> To: [email protected] <[email protected]> > >> Subject: [DISCUSS]The new Plan of global params External Email > >> > >> > >> > >> Hi Dev Team > >> > >> The scheme of The global params that Task need change as blow。 > >> Here is the ISSUE about this DISCUSS。 > >> > >> > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fdolphinscheduler%2Fissues%2F5565&data=04%7C01%7Cweruan%40ebay.com%7C71c6b15115994fd6463808d924cdbf7b%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637581288947116284%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Na6C8tAQODl%2BrfThuaGmtCcVqqwt1eSBpQTFgaRuVXc%3D&reserved=0 > >> < > >> > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fdolphinscheduler%2Fissues%2F5565&data=04%7C01%7Cweruan%40ebay.com%7C2a73397f8b394100b21508d924d28b37%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637581309371279320%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=1B%2BKIBz%2BCfhTp0nRiK3oKFz4KKgYQDN%2B46xF091o0BQ%3D&reserved=0 > >> > > >> > >> I will change the scheme of global param that depend on the relation of > >> the task. > >> Here is the detail about this . > >> > >> 1. create taskInstance > >> > >> Get the previous tasks ,get the varPool of those tasks, put those into > >> this varPool. If the previous task has the same varPool name,use the > value > >> that is not null.If all of the values are null, use the earlier one. > >> > >> 1. Worker get the param > >> > >> Master will send the varPool to the Worker ,and taskPorcessor get the > >> varPool with the format of List<Property>, varPool will do the same as > the > >> localParam. > >> > >> 1. Worker response out Parm > >> > >> When the user define the OUT param in the page of Task Definition.Worker > >> will get the result of Processor . > >> > >> The different Processor return the different format ,for example SQL > >> return the format of List<Map<String,String>>,users could get more than > one > >> line or more than one column;SHELL get the Map<String,String> or > >> String.This out Param will add into the varPool ,and send to the Master > >> ,save into the databases.Also ,this value will save into the localParam > >> > >> > >> > >> If you have some question or have the better plan ,please contact me > >> .thank you. > >> > >> 如果您错误接收了该邮件，请通过电子邮件立即通知我们。请回复邮件到 [email protected] > ，即可以退订此邮件。我们将立即将您的信息从我们的发送目录中删除。 > >> If you have received this email in error please notify us immediately by > >> e-mail. Please reply to [email protected] ,you can unsubscribe > from > >> this mail. We will immediately remove your information from send > catalogue > >> of our. > >> 如果您错误接收了该邮件，请通过电子邮件立即通知我们。请回复邮件到 [email protected] > ，即可以退订此邮件。我们将立即将您的信息从我们的发送目录中删除。 > >> If you have received this email in error please notify us immediately by > >> e-mail. Please reply to [email protected] ,you can unsubscribe > from > >> this mail. We will immediately remove your information from send > catalogue > >> of our. > >> >
