boy-xiaozhang commented on issue #7156: URL: https://github.com/apache/dolphinscheduler/issues/7156#issuecomment-986216738
hi @zhongjiajie , let me answer your question. 1,The one question: > But it seem you missing task cccc in the graph showed below. Did I missing some information or you missing some task graph? in this example, ```sql INSERT OVERWRITE TABLE test.aaa SELECT id FROM test.bbb; INSERT OVERWRITE TABLE test.bbb SELECT id FROM test.cccc; INSERT OVERWRITE TABLE test.dddd SELECT id FROM test.aaa left join test.fff ``` The ccc table may be a table extracted by the user, such as the ods layer, but there are many types of extracted frameworks, such as seaTunnel, dataX, sqoop, flume or flinkX. This kind of task requires us to customize the blood relationship analysis. In order to perform the extraction task auto dag, but whether it is worthwhile for dolphinscheduler, this question may be worth communicating; 2,The second question: > or For the existing dag, the sql content is parsed when the user saves the task,task content is this function is really good, better than I thought, we can indeed rely on it to do the analysis, and give the upstream tasks that users recommend to rely on, minus the cost of users switching back and forth to search; 3,Regarding the above antlr or calcite question: > what does this mean? What directly jar package? did you mean we should use Antlr or calcite ? This means which way we should parse user SQL, spark and hive built-in parse class or antlr4 or druid i prefer the built-in parse class, antlr 4 is also good, I don't know much about calcite -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
