Hi, Just tested with OWB. OWB is faster, but there is still an overhead of 2.5 (instead of 3). Here are my results: | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter 10240 | |====================================================================================================================================================| | DS| 0.029778038| 0.068785804| 0.17412749 | 0.386697864| 0.349159707| 0.7658348 | 1.664042074| 2.957301103| 5.538809715| 12.854707047| 22.860419748| | EM| 0.011113307| 0.018919042| 0.071415593| 0.080011252| 0.238572129| 0.309746715| 0.6318897 | 1.533790385| 2.543436744| 5.104184549 | 10.577758559|
Grüße Johannes -----Original Message----- From: Thomas Andraschko [mailto:[email protected]] Sent: Thursday, June 15, 2017 3:35 PM To: [email protected] Subject: Re: Performance of DeltaSpike Data Hi, It would be great if you could try your sample with OWB instead of weld. I always to such tests with OWB. Regards, Thomas Am Donnerstag, 15. Juni 2017 schrieb Schäfer, Johannes : > Hi, > > Thanks for your investigation. I tried it today again with the last > version from Github. For my test setup I still have the same numbers. > So in average DeltaSpike needs 3 times longer that a pure EM > implementation for finding entities by a query. > For now I stopped my evaluation of DeltaSpike Data. My personal > feeling is quite mixed. I really like to concept of DeltaSpike Data, > but the performance loss is sometimes quite too big. > I hope that there will be in future some improvements. > > Regards > Johannes > > -----Original Message----- > From: Thomas Andraschko [mailto:[email protected] > <javascript:;> ] > Sent: Friday, June 9, 2017 4:40 PM > To: [email protected] <javascript:;> > Subject: Re: Performance of DeltaSpike Data > > FYI: > > i created a similar example based on the unittests in the ds data module. > Here is the result of 100.000 selects, running on tomee and hsqldb: > > DS: 2600ms > JPA: 1800ms > > > 200-300ms are taken by our interceptor lookup, which can be improved a > bit by caching the information about interceptors in the proxy > instance instead of doing a bean+cache lookup each time. > another 200-300ms are taken by DS Data directly - not sure if we can > improve it further. > > Not sure about the other 200-400ms. > > 2017-06-08 16:33 GMT+02:00 Schäfer, Johannes <[email protected] > <javascript:;>>: > > > No chances with @ApplicationScoped. > > To run my test just run "mvn clean install -PEmbeddedTestWeld". > > It is using a in memory H2 DB. Maybe pipe the output into a log to > > find the test results in the output. > > > > Grüße > > Johannes > > > > > > -----Original Message----- > > From: Thomas Andraschko [mailto:[email protected] > <javascript:;>] > > Sent: Thursday, June 8, 2017 3:12 PM > > To: [email protected] <javascript:;> > > Subject: Re: Performance of DeltaSpike Data > > > > Could you also try to make your repository @ApplicationScoped? > > > > How can i run your tests? I don't like to install a AS or database > > ;) > > > > 2017-06-08 13:28 GMT+02:00 Schäfer, Johannes <[email protected] > <javascript:;>>: > > > > > Same with a locally compile version. > > > Mysql: > > > ____________________________________________________________ > > > ____________________________________________________________ > > > _____________________________ > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter > > > 10240 | > > > |=========================================================== > > > ============================================================ > > > =============================| > > > | DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252| > > > | DS| 0.385907351| > > > 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392| > > > 25.372525787| > > > | EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573| > > > | EM| 0.195617863| > > > 0.386509868| 0.812829151| 1.4044238 | 3.007676477| 6.232350452 | > > > 11.726264467| > > > > > > Grüße > > > Johannes > > > > > > > > > -----Original Message----- > > > From: Thomas Andraschko [mailto:[email protected] > <javascript:;>] > > > Sent: Thursday, June 8, 2017 1:12 PM > > > To: [email protected] <javascript:;> > > > Subject: Re: Performance of DeltaSpike Data > > > > > > please build DS from source, i don't think that SNAPSHOT is up to date. > > > > > > 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <[email protected] > <javascript:;>>: > > > > > > > Hi, > > > > > > > > Thanks for your great support. Now I had the time to run the tests. > > > > Unfortunately no improvement. :-( I used Mysql and H2 and both > > > > still have a significant difference. > > > > mysql: > > > > ____________________________________________________________ > > > > ____________________________________________________________ > > > > _____________________________ > > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 > | > > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | > iter > > > > 10240 | > > > > |=========================================================== > > > > ============================================================ > > > > =============================| > > > > | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317| > > > > | DS| 0.489673972| > > > > 1.000932095| 1.418196146| 3.396942334| 6.268094687| > > > > 12.142304859| 24.631240985| > > > > | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944| > > > > | EM| 0.15662194 > > > > | EM| | > > > > 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 > > > > | 10.856276852| > > > > > > > > h2: > > > > ____________________________________________________________ > > > > ____________________________________________________________ > > > > __________________________ > > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 > | > > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter > > > 10240 | > > > > |=========================================================== > > > > ============================================================ > > > > ==========================| > > > > | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493| > > > > | DS| 0.037355253| > > > > 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893| > > > > 0.848326297| > > > > | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262| > > > > | EM| 0.003144109| > > > > 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248| > > > > 0.217664259| > > > > > > > > I used version 1.8.1-SNAPSHOT for testing. > > > > See https://github.com/johannesschaefer/javaee_jsf_ > > > > cdi_jpa_data_ds_project_template > > > > > > > > Grüße > > > > Johannes > > > > > > > > -----Original Message----- > > > > From: Thomas Andraschko [mailto:[email protected] > <javascript:;>] > > > > Sent: Thursday, June 8, 2017 10:36 AM > > > > To: [email protected] <javascript:;> > > > > Subject: Re: Performance of DeltaSpike Data > > > > > > > > I did a major improvement and in my tests, both plain JPA and DS > > > > Data are now very similar. > > > > Would be great if you could provide the new numbers. > > > > > > > > 2017-06-07 14:33 GMT+02:00 Thomas Andraschko > > > > <[email protected] <javascript:;> > > > > >: > > > > > > > > > Hi, > > > > > > > > > > could you please try to run your test again against the github > > master? > > > > > I already did a small improvement and refactored a little bit > > > > > on the weekend. > > > > > > > > > > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <[email protected] > <javascript:;>>: > > > > > > > > > >> Hi, > > > > >> > > > > >> So after the a long weekend, I'm back with my results. > > > > >> For the write, findByPK and findAll tests I get now good numbers. > > > > >> See: > > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ > > > > >> ds_project_template/blob/master/src/test/java/de/psi/ > > > > >> metals/futurelab/repo/benchmark/SaveTest.java > > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ > > > > >> ds_project_template/blob/master/src/test/java/de/psi/ > > > > >> metals/futurelab/repo/benchmark/ReadTest.java > > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ > > > > >> ds_project_template/blob/master/src/test/java/de/psi/ > > > > >> metals/futurelab/repo/benchmark/ReadAllTest.java > > > > >> > > > > >> The difference between delta spike and plain EM are just a > > > > >> few percent, in both directions ;-) . > > > > >> > > > > >> But I wrote a new test case were I try to find entities by an > query. > > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ > > > > >> ds_project_template/blob/master/src/test/java/de/psi/ > > > > >> metals/futurelab/repo/benchmark/ReadQueryTest.java > > > > >> So I compare > > > > >> TypedQuery< Material > query = eml.createQuery( > > > > >> "SELECT m FROM Material m WHERE grade = > > > > >> :grade AND width = :width AND thickness = :thickness", > > > > >> Material.class ); > > > > >> query.setParameter( "grade", "AAA" ); > > > > >> query.setParameter( "width", 5 ); > > > > >> query.setParameter( "thickness", 5. ); List< > > > > >> Material > > > > >> > mats = query.getResultList(); > > > > >> > > > > >> to > > > > >> List< Material > mats = > > > > >> matRepo.findByGradeAndWidthAndThickness( > > > > >> "AAA", 5, 5. ); > > > > >> > > > > >> Here again the difference is quite high. > > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 > > | > > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | > > iter > > > > >> 10240 | > > > > >> |=========================================================== > > > > >> ============================================================ > > > > >> =============================| > > > > >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952| > > > > >> | DS| 0.526700787| > > > > >> 1.023574545| 1.806960223| 3.426772405| 6.969935385| > > > > >> 13.963582543| 26.785764953| > > > > >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918| > > > > >> | EM| 0.171045079| > > > > >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| > > > > >> 6.446844753 > > > > >> | 12.361550486| > > > > >> > > > > >> So as you can see the DeltaSpike implementation needs at > > > > >> least the double amount of time. > > > > >> > > > > >> Any hints for improving the performance? > > > > >> > > > > >> Regards, > > > > >> Johannes > > > > >> > > > > >> -----Original Message----- > > > > >> From: Schäfer, Johannes [mailto:[email protected] > > > > >> <javascript:;>] > > > > >> Sent: Thursday, June 1, 2017 2:27 PM > > > > >> To: [email protected] <javascript:;> > > > > >> Subject: RE: Performance of DeltaSpike Data > > > > >> > > > > >> Right. Copy and paste error. > > > > >> I added also a flush to the EM test. > > > > >> Now I have similar numbers. > > > > >> ____________________________________________________________ > > > > >> ____________________________________________________________ > > > > >> ______________________________ > > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 > > | > > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | > > iter > > > > >> 10240 | > > > > >> |============================================================ > > > > >> |== > > > > >> |== > > > > >> |== > > > > >> |== > > > > >> |=== > > > > >> |============================================================ > > > > >> |== > > > > >> |== > > > > >> |== > > > > >> |== > > > > >> |=== > > > > >> |=======| > > > > >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036| > > > > >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004| > > > > >> | DS| 6.049719288| 34.101109279| 101.589077365| > > > > >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649| > > > > >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769| > > > > >> | EM| 5.960683928| 25.604583163| 106.688041149| > > > > >> > > > > >> It's a little bit strange for me, why I have to compare > > > > >> EntityPersistenceRepository.save with a em.persist + em.flush. > > > > >> I would expect that an simple > > > > >> EntityPersistenceRepository.save don't have a flush (there is > > > > >> a separate > EntityPersistenceRepository. > > > > saveAndFlush). > > > > >> > > > > >> When I do a run with EntityPersistenceRepository.saveAndFlush > > > > >> I get the following numbers. > > > > >> ____________________________________________________________ > > > > >> ____________________________________________________________ > > > > >> ______________________________ > > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 > > | > > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | > > iter > > > > >> 10240 | > > > > >> |============================================================ > > > > >> |== > > > > >> |== > > > > >> |== > > > > >> |== > > > > >> |=== > > > > >> |============================================================ > > > > >> |== > > > > >> |== > > > > >> |== > > > > >> |== > > > > >> |=== > > > > >> |=======| > > > > >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994| > > > > >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685| > > > > >> | DS| 9.902395587| 40.84301017 | 172.179435413| > > > > >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211| > > > > >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412| > > > > >> | EM| 5.842606084| 23.540393571| 132.817886521| > > > > >> > > > > >> So I have the feeling that there is still something wrong. > > > > >> > > > > >> Thanks to Gerhard for his additional hints. > > > > >> I committed all changes to the github repo. > > > > >> > > > > >> Regards, > > > > >> Johannes > > > > >> > > > > >> -----Original Message----- > > > > >> From: Gerhard Petracek [mailto:[email protected] > <javascript:;>] > > > > >> Sent: Thursday, June 1, 2017 1:21 PM > > > > >> To: [email protected] <javascript:;> > > > > >> Subject: Re: Performance of DeltaSpike Data > > > > >> > > > > >> @johannes: > > > > >> as mentioned yesterday you have to move > > > > >> EntityTransaction#begin and EntityTransaction#commit into the > > > > >> for-loop. > > > > >> > > > > >> regards, > > > > >> gerhard > > > > >> > > > > >> > > > > >> > > > > >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko > > > > >> <[email protected] <javascript:;> > > > > >> >: > > > > >> > > > > >> > Hi, > > > > >> > > > > > >> > ~1 year ago i did many optimizations in the data module and > > > > >> > AFAIR DS Data was only a little bit slower. > > > > >> > After i compared my testcase with a benchmark from a user, > > > > >> > the big different came from the transaction handling which > > > > >> > was different in both testcases. > > > > >> > > > > > >> > Regards, > > > > >> > Thomas > > > > >> > > > > > >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek > > > > >> > <[email protected] <javascript:;> > > >: > > > > >> > > > > > >> > > hi johannes, > > > > >> > > > > > > >> > > after refactoring your initial code to ds-test-control i > > > > >> > > saw > > e.g. > > > > >> > > ~6s vs 7,5s for 2560 iterations. > > > > >> > > i'll compare my local version with your new version > > > > >> > > (mentioned in your mail). > > > > >> > > > > > > >> > > regards, > > > > >> > > gerhard > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes > > > > >> > > <[email protected] <javascript:;> > > >: > > > > >> > > > > > > >> > > > Hi, > > > > >> > > > > > > > >> > > > My company is thinking about using DeltaSpike Data. But > > > > >> > > > before we integrate this into our development I was > > > > >> > > > asked to prepare some > > > > >> > > benchmarks, > > > > >> > > > comparing the usage of DeltaSpike Data with the usage > > > > >> > > > of a plain EntityManager. > > > > >> > > > I prepared some benchmarks and I was surprised that > > > > >> > > > there is a big difference in the write performance. I > > > > >> > > > already got some hints in the > > > > >> > > delta > > > > >> > > > spike irc channel, but the performance is still bad. > > > > >> > > > Based on a template from os890 I implemented my tests > > > > >> > > > and prepared a github project. > > > > >> > > > https://github.com/johannesschaefer/javaee_jsf_ > > > > >> > cdi_jpa_data_ds_project_ > > > > >> > > > template > > > > >> > > > Basically I'm talking about this test: > > > > >> > > > https://github.com/johannesschaefer/javaee_jsf_ > > > > >> > cdi_jpa_data_ds_project_ > > > > >> > > > template/blob/master/src/test/java/de/psi/metals/future > > > > >> > > > la b/ repo/benchmark/SaveTest.java > > > > >> > > > > > > > >> > > > It just saves an entity into a DB in a loop. Depending > > > > >> > > > of the number of iterations the difference is quite big. > > > > >> > > > > > > > >> > > > SaveTest > > > > >> > > > _______________________________________________________ > > > > >> > > > __ > > > > >> > > > __ > > > > >> > > > _ > > > > >> > > > _______________________________________________________ > > > > >> > > > __ __ _ _____________________________ > > > > >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | > iter > > > 160 > > > > >> | > > > > >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter > 5120 > > > | > > > > >> iter > > > > >> > > > 10240 | > > > > >> > > > |====================================================== > > > > >> > > > |== > > > > >> > > > |== > > > > >> > > > |= > > > > >> > > > ======================================================= > > > > >> > > > == == = =============================| > > > > >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| > > > > >> > > > | DS| 0.016966639| 0.043319612| > > > > >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818| > > > > >> > > > 20.920141274| 93.631768475| > > > > >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| > > > > >> > > > | EM| 0.004834958| 0.028243393| > > > > >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674| > > > > >> > > > 0.745850523 > > > > >> > > > | > > > > >> > > > 0.853543234 | > > > > >> > > > > > > > >> > > > Also the difference between a run with 5120 and 10240 > > > > >> > > > iteration is not just the double amount of time, it is > > > > >> > > > more than 4 times > > > > more. > > > > >> > > > > > > > >> > > > Do you have some hints to me what I'm doing wrong there? > > > > >> > > > > > > > >> > > > Regards > > > > >> > > > Johannes > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > >
