Thanks for the elaborated explanation, I agree the solution 2 is more flexible, more like a plugin mechanism, and solves my pre/post blocking script expectation.
What I said about the modification to the DAG when one just wants to block the workflow temporarily, that still remains the doubt to me, but I guess if we have to get a conclusion to this solution, it's relatively harmless. Looking forward to seeing the release of this feature! On Thu, Jul 8, 2021 at 11:10 PM 黄智远 <[email protected]> wrote: > First of all, I express my deep gratitude to @simsicon, @dailidong, > @Jave-Chen and @shaojwu. Thank you for your interest in this feature and > your advice! > > So for, the feedback received is more in favor of Solution I (modified > based on existing nodes), but **I think Solution II (adding new node type) > would be more suitable for DS**. I will discuss why I think the second > solution is more suitable from the following two aspects. > > - Functionality (From basic functions and long-term plans) > - Cost (From users and developers) > > ## Functionality > > ### Basic Functions > > In order to choose a solution properly, in my opinion, the baseline is > reaching the basic functions. The goal of blocking task is that in the > execution process in a workflow, **if encountered blocking task and the > task met blocking requirements, the whole workflow should have been paused, > otherwise, continued.** > > Obviously, **two of solutions will reach that goal.** In addition, in some > sicuation, solution II will be better than solution I. Let me show you. > > > Suppose your boss have a demond. There is a blocking task in workflow. > And there are three independent sub_processes in front of the blocking > task. Now, your boss tells you that if any two of the three sub_processes > run successfully, the workflow continue, or else, pause. > > If I choose solution I, I will use condition node. In the current version > of condition node, I must have two branches, one for success, one for > failure, and two branch must be different. For completing the blocking > task, **I must have a useless branch** (echo helloword in shell for example > ) to meet my boss requirement. Maybe you will consider to modify current > condition node, but I think it is not suitable. Because you **break the > condition node function** which is used to turn the workflow into the user > expected direction. > > If I choose solution II, everything becomes easy. As for this new type of > node (Let me call it blocking node), we do not have limitations. This node > has logic judgement function and it can support one successor branch or > multi-successors branchs. In this way, **it does not destroy the > functionality of existing nodes and accomplishes our blocking goal.** > > ### Long-term plans > > In solution I, blocking task is **used as a special node attribute**. If > one day, Apache releases a new bigdata component and DS wants to support > it, we still need to add the blocking option when coding related web pages. > But in solution II, put things right once and for all. > > ## Cost > > ### User > > To be honest, the user cost is similar in both solutions. User will know > what blocking task is and how to setup. @simsicon says that solution II > will cause the modification of DAG. Actually, neithor solution I or > solution II needs to modify the DAG, but the degree is different. > > ### Developer > > Coding in solution II maybe harder. But I believe that is worthy. Beacuse > solution II is more flexiable and competent than solution I, just like I > menthoned in the above sicuation. For another instance, Apple releases > iPad Air 4 and iPad Pro 2021. iPad Air 4 (256GB) sales 749 dollars and iPad > Pro (128GB, 11 inch) sales 749 dollars. They differ by $50. But with that > extra money you can enjoy the ultimate iPad experience while meeting your > daily needs, why not pay that extra price? Blocking task is the same. > > In conclution, I think solution II is better. > > Welcome to express your opinion! > > =================== > Stay Hungry, Stay Foolish > Martin Huang > [email protected] > Sent with a Spark > =================== > -- Thanks&Best Regards Zhang Zheng
