ad1happy2go commented on issue #10697:
URL: https://github.com/apache/hudi/issues/10697#issuecomment-1953554106
@alberttwong Thanks for trying all that. I issue here is not the code but
memory. I guess you will be able to run same code with small parquet size.
Can you try to do df.write.format("parquet").save("some test parquet path")
and see if that works.
If that also fails, then certainly this file is too big to process in
available resources.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
