dabla commented on PR #42572: URL: https://github.com/apache/airflow/pull/42572#issuecomment-2518245561
> I'll go back to the list with this feedback, but this implementation is tantamount to a parallel scheduler, executor and triggerer implementation so is very unlikely to be accepted in to core. Yes, that's because you just can't simply execute multiple deferrable operator within a loop by calling their execute method. Maybe we could think of some kind of interface/expansion mechanism to the PartialOperator that you could enhance with additional functionalities, like here the iterate method which returns an IterableOperator instead of a MappedOperator. Then we could define this functionality into a provider, then it doesn't have to be part of the core for example if this would be the ultimate issue. Anyway I've started drafting an AIP with @Joffreybvn where we describe what we want to solve here and also trying to summarize everything we've discussed so far here, in slack and devlists. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
