VitoMakarevich commented on issue #11122: URL: https://github.com/apache/hudi/issues/11122#issuecomment-2408936629
Maybe the workaround can be set up - so that the archived timeline is loaded when a certain new flag is enabled(false by default) - this way it can be done, although expensive, but possible. Or clean can be revisited to split a number of partitions into reasonable numbers - but I'm sure it is impossible as cleaning 2M partitions will exhaust driver memory... Currently, it's simply not possible to run clean if for any reason it was disabled for some time so the last clean is archived. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
