SeeMeJuiner opened a new issue, #15115:
URL: https://github.com/apache/dolphinscheduler/issues/15115

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and 
found no similar feature requirement.
   
   
   ### Description
   
   I used dolphinscheduler for a few months now, I would like to offer a few 
suggestions for discussion:
   Question 1: Parameter passing
   In dolphinshceduler, task level parameters and workflow level parameters are 
supported, but I found that some parameter sets could not be reused throughout 
the project or even across workflows;
   In view of the above requirements, I hope to add the following optimization 
in the future:
   1. Correlation between workflows [sub_process] supports parameter passing. 
If I define the pre - and post-position relationships of multiple workflow in a 
sub_process task, the parameter list I defined in the first place can be reused 
in the current task execution cycle.
   2. Reuse of the whole project-level parameter set (note the set), similar to 
shell or sql task support parameter definition;
   In general, our scheduling tools in the real world use more reusable 
parameters, so that the parameters are not limited to a single job flow or job;
   We first need to define a general parameter set script, through each run 
batch automatically get the latest parameters only, I only need to use the 
parameters where the introduction of my dependency script can achieve parameter 
reuse, this is really very traversal.
   
   
   Question 2: Event monitoring
   The event monitoring function is introduced in the scheduling policy to 
support multiple job triggering modes.
   In the actual scenario, we need to interact with other systems to achieve 
the automatic triggering and flow of job batches, and usually the more common 
interaction is the tag file (such as.OK file),
   The presence of a tag file indicates that the upstream job or task we depend 
on has been executed and can trigger the post task defined in our system.
   
   
   Question 3: Daily switching mechanism
   At present, there is only timing policy in the system to realize date 
switching.
   Often when the system job runs batch, there may be some delays, errors and 
other problems that may lead to the overall delay of the day's job execution 
batch;
   If the batch is cross-dated, some of the previous day's jobs may have missed 
execution, which can lead to data errors.
   
   ### Use case
   
   _No response_
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: 
[email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to