kbendick opened a new pull request #2595: URL: https://github.com/apache/iceberg/pull/2595
Right now, we have JMH benchmarks to allow performance comparisons between the native spark sources / sinks vs the iceberg sources and sinks in spark. However, these JMH test suites don't get registered for Spark 3. I have moved the JMH test code to `spark/src/jmh/...` and then added that to the src directories for those projects. Now, depending on which JDK one is using, you can run either Spark 2 tests (with JDK 8) or Spark 3 tests (with JDK 11). It would still be possible to add specific benchmarks to Spark 3 or Spark 2 by placing them in `spark[2|3]/src/jmh/...`. For now, all of the code is left as is and has been copied over (with the command to run updated in each file). I've also added a `.gitkeep` file in `spark2/benchmark` and `spark3/benchmark` so that users can run the commands as is without having to create the folders. Based on the way the tests are currently run, it would be difficult to create the folder and the given name per test without a larger refactoring, and I wanted to keep this change as small as possible. This closes https://github.com/apache/iceberg/issues/2590 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
