yanghua commented on pull request #2868: URL: https://github.com/apache/hudi/pull/2868#issuecomment-828408491
> > I have two questions: > > > > 1. The lowest Flink version we supported is 1.12.x? > > 2. Can we provide an e2e demo and documentation to show the usage of the flink streamer via jar-mode, just like delta-streamer, it should be out of the box; > > > > I tried it, but missed the dependencies of the Kafka connector. Can we make the new flink streamer peer to the delta streamer? > > Yes, people would only use flink 1.12.x code, the code to remove is not because of flink version, it's because the logic is totally redundant. Remove to avoid cofusion, because i found some people use the legacy code with poor performance. Although I know that many users are currently testing based on 1.12, the threshold we set for many users of older versions is very high. Pray that they are willing to upgrade the Flink version in order to use hudi. In fact, I personally think that the biggest improvement of the new implementation lies in the bucket assigner. As for other points, we could have found a solution (although it does not seem very elegant). Well, I don't have to worry about the Flink version anymore, and I don't have time to pay attention to the old implementation. > I still think we should not include a kafka connector into the delta streamer, on one complains the missing of it, based on the users i see. I still insist that we need to include kafka-related dependencies. If you look back at the HoodieFlinkStreamerV2 class. What is it in essence? It is just a program written using Flink DataStream API, which is specific (Kafka -> Hudi), not plug-in-oriented or abstract-oriented. For a specific Flink program, we should provide users with an Uber(fat) Jar. Instead of letting users pay attention to details and pay additional costs. Otherwise, why don't we make the source universal? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
