yangyichao-mango commented on issue #3072: URL: https://github.com/apache/incubator-dolphinscheduler/issues/3072#issuecomment-658108391
> I think, the main responsibilities of DS are scheduling and job execution, Data processing should be the responsibility of the node, E.g datax. > […](#) > -------------------- DolphinScheduler(Incubator) Commtter Hemin Wen 温合民 [email protected] -------------------- leehom <[email protected]> 于2020年7月10日周五 上午9:10写道： > ds can be seen as a data processor, sharding can improving the performance in whole data processing lifecycle. for example, data input from db table, sharding task can input data from table Parallizing; the save happening at data converting , udf , data wrriting to db — You are receiving this because you commented. Reply to this email directly, view it on GitHub <[#3072 (comment)](https://github.com/apache/incubator-dolphinscheduler/issues/3072#issuecomment-656426207)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJNXTBOUQVHK7VXFIFWFQWDR2ZTCBANCNFSM4OK5W7VA> . I agree with @Rubik-W . Can you describe the benefit of implementing sharding or paralleling task in scheduler engine. I think the responsibility of the scheduling engine is scheduling, and sharding should be within the scope of the data processing engine. Perhaps in most cases, the performance of sharding on the scheduling system will not be better than that on the data processing engine. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
