soumilshah1995 commented on issue #8260:
URL: https://github.com/apache/hudi/issues/8260#issuecomment-1483881585
I'm not sure if I get your query completely, however RFC 51 gives you a
means to access changes occurring on your data lake.
INSERT | UPDATE | DELETE example You can power your downstream pipeline by
capturing these CDC events. Well, let me say that it's similar to Debezium,
giving you access to all Datalake modifications.
However, based on the diagram you provided and what I have read so far, I
would conclude that your main goal is to obtain changes that have occurred in
your Transactional Data Lake, query them incrementally, and conduct left outer
join with RAW source.
Actually, once I have some free time, I plan to try these architectural
designs; they are on my list of things to do.
I'm happy to connect with you on Slack and talk more with you on the Hudi
Slack channel. :D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
