please build DS from source, i don't think that SNAPSHOT is up to date. 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <[email protected]>:
> Hi, > > Thanks for your great support. Now I had the time to run the tests. > Unfortunately no improvement. :-( > I used Mysql and H2 and both still have a significant difference. > mysql: > ____________________________________________________________ > ____________________________________________________________ > _____________________________ > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter > 10240 | > |=========================================================== > ============================================================ > =============================| > | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317| 0.489673972| > 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859| > 24.631240985| > | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944| 0.15662194 | > 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 | > 10.856276852| > > h2: > ____________________________________________________________ > ____________________________________________________________ > __________________________ > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 | > |=========================================================== > ============================================================ > ==========================| > | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493| 0.037355253| > 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893| 0.848326297| > | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262| 0.003144109| > 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248| 0.217664259| > > I used version 1.8.1-SNAPSHOT for testing. > See https://github.com/johannesschaefer/javaee_jsf_ > cdi_jpa_data_ds_project_template > > Grüße > Johannes > > -----Original Message----- > From: Thomas Andraschko [mailto:[email protected]] > Sent: Thursday, June 8, 2017 10:36 AM > To: [email protected] > Subject: Re: Performance of DeltaSpike Data > > I did a major improvement and in my tests, both plain JPA and DS Data are > now very similar. > Would be great if you could provide the new numbers. > > 2017-06-07 14:33 GMT+02:00 Thomas Andraschko <[email protected] > >: > > > Hi, > > > > could you please try to run your test again against the github master? > > I already did a small improvement and refactored a little bit on the > > weekend. > > > > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <[email protected]>: > > > >> Hi, > >> > >> So after the a long weekend, I'm back with my results. > >> For the write, findByPK and findAll tests I get now good numbers. > >> See: > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ > >> ds_project_template/blob/master/src/test/java/de/psi/ > >> metals/futurelab/repo/benchmark/SaveTest.java > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ > >> ds_project_template/blob/master/src/test/java/de/psi/ > >> metals/futurelab/repo/benchmark/ReadTest.java > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ > >> ds_project_template/blob/master/src/test/java/de/psi/ > >> metals/futurelab/repo/benchmark/ReadAllTest.java > >> > >> The difference between delta spike and plain EM are just a few > >> percent, in both directions ;-) . > >> > >> But I wrote a new test case were I try to find entities by an query. > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ > >> ds_project_template/blob/master/src/test/java/de/psi/ > >> metals/futurelab/repo/benchmark/ReadQueryTest.java > >> So I compare > >> TypedQuery< Material > query = eml.createQuery( > >> "SELECT m FROM Material m WHERE grade = :grade AND > >> width = :width AND thickness = :thickness", > >> Material.class ); > >> query.setParameter( "grade", "AAA" ); > >> query.setParameter( "width", 5 ); > >> query.setParameter( "thickness", 5. ); List< Material > > >> mats = query.getResultList(); > >> > >> to > >> List< Material > mats = matRepo.findByGradeAndWidthAndThickness( > >> "AAA", 5, 5. ); > >> > >> Here again the difference is quite high. > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter > >> 10240 | > >> |=========================================================== > >> ============================================================ > >> =============================| > >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952| > >> | DS| 0.526700787| > >> 1.023574545| 1.806960223| 3.426772405| 6.969935385| 13.963582543| > >> 26.785764953| > >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918| > >> | EM| 0.171045079| > >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 | > >> 12.361550486| > >> > >> So as you can see the DeltaSpike implementation needs at least the > >> double amount of time. > >> > >> Any hints for improving the performance? > >> > >> Regards, > >> Johannes > >> > >> -----Original Message----- > >> From: Schäfer, Johannes [mailto:[email protected]] > >> Sent: Thursday, June 1, 2017 2:27 PM > >> To: [email protected] > >> Subject: RE: Performance of DeltaSpike Data > >> > >> Right. Copy and paste error. > >> I added also a flush to the EM test. > >> Now I have similar numbers. > >> ____________________________________________________________ > >> ____________________________________________________________ > >> ______________________________ > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter > >> 10240 | > >> |==================================================================== > >> |=== > >> |==================================================================== > >> |=== > >> |=======| > >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036| > >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004| > >> | DS| 6.049719288| 34.101109279| 101.589077365| > >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649| > >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769| > >> | EM| 5.960683928| 25.604583163| 106.688041149| > >> > >> It's a little bit strange for me, why I have to compare > >> EntityPersistenceRepository.save with a em.persist + em.flush. I > >> would expect that an simple EntityPersistenceRepository.save don't > >> have a flush (there is a separate EntityPersistenceRepository. > saveAndFlush). > >> > >> When I do a run with EntityPersistenceRepository.saveAndFlush I get > >> the following numbers. > >> ____________________________________________________________ > >> ____________________________________________________________ > >> ______________________________ > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter > >> 10240 | > >> |==================================================================== > >> |=== > >> |==================================================================== > >> |=== > >> |=======| > >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994| > >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685| > >> | DS| 9.902395587| 40.84301017 | 172.179435413| > >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211| > >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412| > >> | EM| 5.842606084| 23.540393571| 132.817886521| > >> > >> So I have the feeling that there is still something wrong. > >> > >> Thanks to Gerhard for his additional hints. > >> I committed all changes to the github repo. > >> > >> Regards, > >> Johannes > >> > >> -----Original Message----- > >> From: Gerhard Petracek [mailto:[email protected]] > >> Sent: Thursday, June 1, 2017 1:21 PM > >> To: [email protected] > >> Subject: Re: Performance of DeltaSpike Data > >> > >> @johannes: > >> as mentioned yesterday you have to move EntityTransaction#begin and > >> EntityTransaction#commit into the for-loop. > >> > >> regards, > >> gerhard > >> > >> > >> > >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko > >> <[email protected] > >> >: > >> > >> > Hi, > >> > > >> > ~1 year ago i did many optimizations in the data module and AFAIR > >> > DS Data was only a little bit slower. > >> > After i compared my testcase with a benchmark from a user, the big > >> > different came from the transaction handling which was different in > >> > both testcases. > >> > > >> > Regards, > >> > Thomas > >> > > >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek <[email protected]>: > >> > > >> > > hi johannes, > >> > > > >> > > after refactoring your initial code to ds-test-control i saw e.g. > >> > > ~6s vs 7,5s for 2560 iterations. > >> > > i'll compare my local version with your new version (mentioned in > >> > > your mail). > >> > > > >> > > regards, > >> > > gerhard > >> > > > >> > > > >> > > > >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes <[email protected]>: > >> > > > >> > > > Hi, > >> > > > > >> > > > My company is thinking about using DeltaSpike Data. But before > >> > > > we integrate this into our development I was asked to prepare > >> > > > some > >> > > benchmarks, > >> > > > comparing the usage of DeltaSpike Data with the usage of a > >> > > > plain EntityManager. > >> > > > I prepared some benchmarks and I was surprised that there is a > >> > > > big difference in the write performance. I already got some > >> > > > hints in the > >> > > delta > >> > > > spike irc channel, but the performance is still bad. > >> > > > Based on a template from os890 I implemented my tests and > >> > > > prepared a github project. > >> > > > https://github.com/johannesschaefer/javaee_jsf_ > >> > cdi_jpa_data_ds_project_ > >> > > > template > >> > > > Basically I'm talking about this test: > >> > > > https://github.com/johannesschaefer/javaee_jsf_ > >> > cdi_jpa_data_ds_project_ > >> > > > template/blob/master/src/test/java/de/psi/metals/futurelab/ > >> > > > repo/benchmark/SaveTest.java > >> > > > > >> > > > It just saves an entity into a DB in a loop. Depending of the > >> > > > number of iterations the difference is quite big. > >> > > > > >> > > > SaveTest > >> > > > ____________________________________________________________ > >> > > > ____________________________________________________________ > >> > > > _____________________________ > >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 > >> | > >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | > >> iter > >> > > > 10240 | > >> > > > |=========================================================== > >> > > > ============================================================ > >> > > > =============================| > >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639| > >> > > > | DS| 0.043319612| > >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818| > >> > > > 20.920141274| 93.631768475| > >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958| > >> > > > | EM| 0.028243393| > >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674| 0.745850523 > >> > > > | > >> > > > 0.853543234 | > >> > > > > >> > > > Also the difference between a run with 5120 and 10240 iteration > >> > > > is not just the double amount of time, it is more than 4 times > more. > >> > > > > >> > > > Do you have some hints to me what I'm doing wrong there? > >> > > > > >> > > > Regards > >> > > > Johannes > >> > > > > >> > > > > >> > > > > >> > > > >> > > >> > > > > >
