luyongbiao commented on issue #8416: URL: https://github.com/apache/hudi/issues/8416#issuecomment-1502727733
> The reason for above error is when Java programs generated using Catalyst from programs using DataFrame and Dataset are compiled into Java bytecode, the size of byte code of one method must not be 64 KB or more. > > You can try setting - spark.sql.codegen.wholeStage= "false" > > Also, I have seen similar unusual behaviour with spark handling large number of fields with other file formats too in past. This setting just worked for the error displays, data was still lost. Do you have Any suggestion for me to fix this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
