weixiaonan1 commented on issue #12953: URL: https://github.com/apache/dolphinscheduler/issues/12953#issuecomment-1323164647
@Radeity Thanks for your feedback. Currently, our alert group just aims to notify the user. It means the total end of a workflow execution. The same goes for sending SQL query results through emails. So if we integrate MQ support into DS, sending these msgs to MQ in order to consume in other systems, it's really quite different from current mode and looks weird. However, from the persperctive of DS itself, we just make limited extension and no changes to existing functions . Add alert-plugin-MQ and send workflow execution result to MQ is same to send it througn DingTalk or emails. DS finish its task when the msgs are sent. The only difference is that user can deal with these msg more flexible. DS doesn't need to know the subsequent process in third-part systems, it's completely decoupled. It's same when we send SQL query results to MQ, DS just needs to send msgs to MQ like sending througn mails now. Users can write msgs in their own db or send a more complex email after processing. The two ways you mentioned in reply are not so good i think. It brings complexities when we use CDC middleware in third-part systems and this way is too heavy. Use http-alert-plugin increases the coupling between DS and third-part systems. In this scenario, MQ is a great solution. Of course, we need to see whether many people have similar requirements to decide whether providing these MQ-related features in DS. And I'm very glad to participate in the development. 现在DS提供的告警组只是向用户**通知**结果，通知结束也就意味着整个工作流执行**闭环**了。SQL任务组件中的查询结果发送邮件同理。引入了消息队列后，打破了这一初衷，执行结果或者查询结果写入消息队列，会有其他系统进行消费，看起来确实有些古怪。但是站在DS的角度来讲，其实完全没有影响现有的逻辑，只是进行了有限的拓展。增加MQ告警组，将执行结果通过MQ发送给用户，和通过钉钉、邮件发送给用户是一样的，通知结束那么DS的任务就完成了，只是用户可以通过消费MQ中的消息来更灵活处理执行结果，至于如何处理完全与DS解耦，DS并不需要关心。SQL任务组件中的查询结果发送给MQ也同理，用户可以利用MQ将查询结果写入自己的数据库、或者加工处理后发送一封更复杂的邮件等等，DS只需要像现在发送邮件一样将查询结果写入MQ就可以 了。 上面提供的两个方案，利用CDC(Canal等)中间件，极大的增加了复杂度；使用http-alert-plugin一定程度上使DS与集成系统耦合紧密。在我所述的两个场景下，使用MQ的方案会更好。 当然DS是否提供这两个特性，需要看看这些需求是特例还是共性，如果很多人有这方面的需要，我很乐意参与开发当中。希望大家多多讨论。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
