gatorsmile commented on a change in pull request #28081: [SPARK-31315][SQL]
SQLQueryTestSuite: Display the total compile time for generated java code
URL: https://github.com/apache/spark/pull/28081#discussion_r410655567
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
##########
@@ -1309,6 +1309,23 @@ object CodeGenerator extends Logging {
// bytecode instruction
final val MUTABLESTATEARRAY_SIZE_LIMIT = 32768
+ // The java source code generated by whole codegen is compiled on the Driver
side and sent to
+ // each Executor as the execution code to process data. This is very
effective in processing
+ // large amounts of data in a distributed environment. However, in the test
environment,
+ // because the amount of data is not large or not executed in parallel, the
compilation time
+ // of these Java source code will become a major part of the entire test
runtime. When
+ // running test cases, we summarize the total compilation time and output it
to the execution
+ // log for easy analysis and view.
+ private val _compileTime = new LongAccumulator
+
+ // Returns the total compile time of Java source code in nanoseconds.
+ // Visible for testing
+ def compileTime: Long = _compileTime.sum
+
+ // Reset compile time.
+ // Visible for testing
+ def resetCompileTime: Unit = _compileTime.reset()
Review comment:
Methods should be declared with parentheses, unless they are accessors that
have no side-effect. See:
https://github.com/databricks/scala-style-guide#parentheses
Fixed in https://github.com/apache/spark/pull/28252
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
