soumilshah1995 commented on issue #8260: URL: https://github.com/apache/hudi/issues/8260#issuecomment-1483881419
I'm not sure if I get your query completely, however RFC 51 gives you a means to access changes occurring on your data lake. INSERT | UPDATE | DELETE example You can power your downstream pipeline by capturing these CDC events. Well, let me say that it's similar to Debezium, giving you access to all Datalake modifications. However, based on the diagram you provided and what I have read so far, I would conclude that your main goal is to obtain changes that have occurred in your Transactional Data Lake, query them incrementally, and conduct left outer join with RAW source. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
