hi, any progress? Do we need a meeting to solve this? By the way, the picture can't show in the apache mailing list, you can upload the pic to github, then paste the url address to the mail.
Best Regards --------------- DolphinScheduler PMC Lidong Dai [email protected] --------------- On Tue, Jun 1, 2021 at 5:04 PM Xingjie Wang(联通集团联通数字科技有限公司) < [email protected]> wrote: > This really doesn't satisfy those scene. > For the first one ,If we want to do this, we should save the varPool to > the level of processInstance, so the Task4 can get the varPool from Task1 > and do not by Task2 and Task3. > This one will obscure the relation between globalParam and localParam and > varPool. > Other plan ,when user define the Task4 IN param ,user should chonse the > Task1 that this param is the Task1's OUT param .When init the Task4's > varPool ,get the Task1 form completeTaskList,then get the varPool.And when > the taskInstance have a new property, mark this taskInstance'name into this > property ,and put this property into the varPool. > It will satisfy those scene. > How do you think? > -----邮件原件----- > 发件人: Ruan, Wenjun <[email protected]> > 发送时间: 2021年6月1日 16:07 > 收件人: [email protected] > 主题: Re: [DISCUSS]The new Plan of global params > > Sorry, it seems that the picture cannot be displayed well, the dag > structure is as follow: > > Task1 -> Task2 -> Task3 -> Task4 > > From: Ruan, Wenjun <[email protected]> > Date: Tuesday, June 1, 2021 at 3:55 PM > To: [email protected] <[email protected]> > Subject: Re: [DISCUSS]The new Plan of global params External Email Hi > Xingjie, > > I have two things want to confirm. > In your plan it seems that we need to store varpool in all the post nodes? > For example, if I have a simple dag like below: > [cid:[email protected]] > > If I want to get the out param of Task1 in Task4, for example we call it > parameterA, then we need to store parameterA in Task2’s varpool and Task3’s > varpool, even if we don’t need parameterA in Task2 and Task3, am I right? > Can we take it directly from Task1? > > The second thing is that if I want to troubleshouting, can I find the > source of the parameters in the varpool? It seems I need to look forward > and find the first node that contains the parameter? It seems not > convenient, if I have much tasks in a dag. > > Thanks, > Wenjun Ruan > > > From: Xingjie Wang(联通集团联通数字科技有限公司) <[email protected]> > Date: Tuesday, June 1, 2021 at 3:21 PM > To: [email protected] <[email protected]> > Subject: [DISCUSS]The new Plan of global params External Email > > > > Hi Dev Team > > The scheme of The global params that Task need change as blow。 > Here is the ISSUE about this DISCUSS。 > > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fdolphinscheduler%2Fissues%2F5565&data=04%7C01%7Cweruan%40ebay.com%7C71c6b15115994fd6463808d924cdbf7b%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637581288947116284%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Na6C8tAQODl%2BrfThuaGmtCcVqqwt1eSBpQTFgaRuVXc%3D&reserved=0 > < > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fdolphinscheduler%2Fissues%2F5565&data=04%7C01%7Cweruan%40ebay.com%7C2a73397f8b394100b21508d924d28b37%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637581309371279320%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=1B%2BKIBz%2BCfhTp0nRiK3oKFz4KKgYQDN%2B46xF091o0BQ%3D&reserved=0 > > > > I will change the scheme of global param that depend on the relation of > the task. > Here is the detail about this . > > 1. create taskInstance > > Get the previous tasks ,get the varPool of those tasks, put those into > this varPool. If the previous task has the same varPool name,use the value > that is not null.If all of the values are null, use the earlier one. > > 1. Worker get the param > > Master will send the varPool to the Worker ,and taskPorcessor get the > varPool with the format of List<Property>, varPool will do the same as the > localParam. > > 1. Worker response out Parm > > When the user define the OUT param in the page of Task Definition.Worker > will get the result of Processor . > > The different Processor return the different format ,for example SQL > return the format of List<Map<String,String>>,users could get more than one > line or more than one column;SHELL get the Map<String,String> or > String.This out Param will add into the varPool ,and send to the Master > ,save into the databases.Also ,this value will save into the localParam > > > > If you have some question or have the better plan ,please contact me > .thank you. > > 如果您错误接收了该邮件，请通过电子邮件立即通知我们。请回复邮件到 > [email protected]，即可以退订此邮件。我们将立即将您的信息从我们的发送目录中删除。 > If you have received this email in error please notify us immediately by > e-mail. Please reply to [email protected] ,you can unsubscribe from > this mail. We will immediately remove your information from send catalogue > of our. > 如果您错误接收了该邮件，请通过电子邮件立即通知我们。请回复邮件到 > [email protected]，即可以退订此邮件。我们将立即将您的信息从我们的发送目录中删除。 > If you have received this email in error please notify us immediately by > e-mail. Please reply to [email protected] ,you can unsubscribe from > this mail. We will immediately remove your information from send catalogue > of our. >
