dmenin commented on issue #3394: URL: https://github.com/apache/hudi/issues/3394#issuecomment-895381894
ok, so bottom line, hudi doesn't have the concept of OLDER and NEWER in terms of row date (timestamp) - it only has NEW and OLD partition (where NEW corresponds to the data being upserted and OLD corresponds to the EXISTING partition of a particular key) If I want the behaviour I described, I probably have to implement myself? Have you been around this use case and can suggest a solution? (the simplest one I can imagine is to manually delete the data thats obsolete and only insert the new data - but to do that, I have to join the incoming data with the existing data and check the differences.... which may not perform in the long term). Thanks for your help so far. Diego -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
