uschindler commented on pull request #177: URL: https://github.com/apache/lucene/pull/177#issuecomment-866054411
Hi @mikemccand, I will open a few more issues in luceneutil. The way how it invokes the JVM is not like it is done in production: - `-Xbatch` as default doesn't help for reproducibility and makes the results problematic for production use, as unrealistic. - `-XX:-TieredCompilation` as default is only useful to run tests (so we don't spare too much time for recompile), but as it is default in modern JDKS since Java 8, it should really be swritched on for benchmarks. While testing this, disabling tiered compilation made results 20% worse because it prevented escape analysis from kicking in early enough. So in general the benchmark should mimic real-live. You are repating the runs several times anyways, so changing the optimizations is not a good idea, because your settings do not make fluctuation better, they slow down! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
