GitHub user rednaxelafx opened a pull request: https://github.com/apache/spark/pull/22103
[SPARK-25113][SQL] Add logging to CodeGenerator when any generated method's bytecode size goes above HugeMethodLimit ## What changes were proposed in this pull request? Add logging for all generated methods from the `CodeGenerator` whose bytecode size goes above 8000 bytes. This is to help with gathering stats on how often Spark is generating methods too big to be JIT'd. It covers all codegen scenarios, include whole-stage codegen and also individual expression codegen, e.g. unsafe projection, mutable projection, etc. ## How was this patch tested? Manually tested that logging did happen when generated method was above 8000 bytes. Also added a new unit test case to `CodeGenerationSuite` to verify that the logging did happen. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rednaxelafx/apache-spark codegen-8k-logging Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22103.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22103 ---- commit 640c9cd3b99d51f38c9b1a0c3f94bae676d11e51 Author: Kris Mok <kris.mok@...> Date: 2018-08-14T10:50:01Z SPARK-25113: Add logging to CodeGenerator when any generated method's bytecode size goes above HugeMethodLimit ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org