This view sounds reasonable, but it is difficult to achive. So developing based on one of these engines is the best choice for now.
Best, Limingtian > 2022年5月7日 17:36，Zongwen Li <[email protected]> 写道： > > The goal of Apache SeaTunnel is different from Apache Beam. > Apache SeaTunnel focuses on source and sink connectors, and develops > features in the field of data integration; > Apache Beam focuses and unifies all the functions of the compute engine, > including operators such as join, connect, map, etc. and it doesn't unify > streaming and batch source. > > This improvement proposal is to solve the current problems encountered by > SeaTunnel . If you have better ideas, you can bring them up for discussion. > > Best, > Zongwen Li > > leo65535 <[email protected]> 于2022年4月29日周五 16:14写道： > >> >> >> Hi @zongwen, >> >> >> I think this is not a good idea, it seems that we will be more and more >> like Apache Beam, >> >> >> Best, >> Leo65535 >> >> >> At 2022-04-18 15:10:08, "李宗文" <[email protected]> wrote: >>> Hi All, >>> In the current implementation of SeaTunnel, the connector is coupled with >>> the computing engine, which results in a connector that needs to be >>> implemented for each engine, and it is difficult to support multiple >>> versions of the engine. >>> >>> Through the questionnaire, it was found that users used multiple versions >>> of Spark and Flink engines, and they also hoped that SeaTunnel would >>> support Change Data Capture (CDC) connectors; >>> >>> Based on the above questions and needs, I created an improvement proposal: >>> https://github.com/apache/incubator-seatunnel/issues/1608 >>> Preliminary idea of Source and Sink API: >>> https://github.com/apache/incubator-seatunnel/issues/1701 >>> https://github.com/apache/incubator-seatunnel/issues/1704 >>> >>> Please discuss away! Zongwen Li >> >
