ad1happy2go commented on issue #8416: URL: https://github.com/apache/hudi/issues/8416#issuecomment-1502707341
The reason for above error is when Java programs generated using Catalyst from programs using DataFrame and Dataset are compiled into Java bytecode, the size of byte code of one method must not be 64 KB or more. You can try setting - spark.sql.codegen.wholeStage= "false" Also, I have seen similar unusual behaviour with spark handling large number of fields with other file formats too in past. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
