Hi, So, as we are leaning towards final, I made another round of benchmarking on HV 6.
They are run on the same machine as the previous "Hibernate Validator 6 benchmarks results" I posted a few months ago [1] so they can be compared to these numbers (you'll notice the 5.4.1 numbers are about the same between the 2 emails). Note that these benchmarks only exercises the validation engine but does not report any violation. I think I'll add some more benchmarks to ensure violation reporting also moves in the right direction. It might also be a good occasion to revisit this benchmark: http://carinae.net/2010/06/benchmarking-hibernate-validator-and-apache-beanvalidation-the-two-jsr-303-implementations/ . With a large value of "revisit" as it's really basic but the scenarios are interesting. Note that I'm a bit pessimistic on the parsing part: we now do a lot more work than before to deal with container elements so the startup cost will probably be significantly higher. == Apache BVal 1.1.2 Result "org.hibernate.validator.performance.simple.SimpleValidation.testSimpleBeanValidation": 357.500 ±(99.9%) 5.327 ops/ms [Average] (min, avg, max) = (337.100, 357.500, 381.472), stdev = 10.760 CI (99.9%): [352.173, 362.827] (assumes normal distribution) Result "org.hibernate.validator.performance.cascaded.CascadedValidation.testCascadedValidation": 379.605 ±(99.9%) 5.817 ops/ms [Average] (min, avg, max) = (360.654, 379.605, 411.361), stdev = 11.750 CI (99.9%): [373.789, 385.422] (assumes normal distribution) (not really clear how BVal happens to be faster in the cascaded validation case) == HV 5.4.1.Final Result "org.hibernate.validator.performance.simple.SimpleValidation.testSimpleBeanValidation": 558.196 ±(99.9%) 3.643 ops/ms [Average] (min, avg, max) = (542.396, 558.196, 575.360), stdev = 7.360 CI (99.9%): [554.552, 561.839] (assumes normal distribution) Result "org.hibernate.validator.performance.cascaded.CascadedValidation.testCascadedValidation": 285.788 ±(99.9%) 1.970 ops/ms [Average] (min, avg, max) = (278.611, 285.788, 298.530), stdev = 3.980 CI (99.9%): [283.817, 287.758] (assumes normal distribution) == Master from March after the first round of improvements (numbers taken from the previous email) Result "org.hibernate.validator.performance.simple.SimpleValidation.testSimpleBeanValidation": 869.546 ±(99.9%) 14.734 ops/ms [Average] (min, avg, max) = (760.007, 869.546, 909.206), stdev = 29.763 CI (99.9%): [854.813, 884.280] (assumes normal distribution) Result "org.hibernate.validator.performance.cascaded.CascadedValidation.testCascadedValidation": 343.699 ±(99.9%) 2.077 ops/ms [Average] (min, avg, max) = (331.333, 343.699, 352.626), stdev = 4.196 CI (99.9%): [341.622, 345.776] (assumes normal distribution) == HV 6 - Current master with https://github.com/hibernate/hibernate-validator/pull/814 applied Result "org.hibernate.validator.performance.simple.SimpleValidation.testSimpleBeanValidation": 924.121 ±(99.9%) 3.686 ops/ms [Average] (min, avg, max) = (905.423, 924.121, 941.295), stdev = 7.446 CI (99.9%): [920.435, 927.807] (assumes normal distribution) Result "org.hibernate.validator.performance.cascaded.CascadedValidation.testCascadedValidation": 430.092 ±(99.9%) 3.661 ops/ms [Average] (min, avg, max) = (416.439, 430.092, 447.607), stdev = 7.396 CI (99.9%): [426.431, 433.754] (assumes normal distribution) == Conclusion The good news is that the results are even better than the ones from March, after some further tweaking. We are significantly faster than BVal in these scenarios and also significantly faster than 5.4.1.Final. Now, we need to get this PR in :). [1] http://lists.jboss.org/pipermail/hibernate-dev/2017-March/016057.html _______________________________________________ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
