GergelyKalmar commented on issue #7909: URL: https://github.com/apache/hudi/issues/7909#issuecomment-1980936824
We eventually abandoned Hudi on AWS/S3 entirely, and we are streaming the changes into a Redshift serverless cluster instead. It might be possible to tweak some configuration options to avoid the throttling or make Spark handle it more gracefully, but we could not make Hudi work reliably with our data sets in the end, despite having help from AWS support engineers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
