Jfyi: I think the performance of the data module is really good but we can improve the proxy module. If the performance is better with owb in your case, the performance will likely be much better with weld after i optimzed the proxy module as we can remove the dynamic bean lookups in the proxy and cache the bean references. Will likely do it in 1-2 weeks.
Am Donnerstag, 15. Juni 2017 schrieb Thomas Andraschko : > Hi, > > It would be great if you could try your sample with OWB instead of weld. > I always to such tests with OWB. > > Regards, > Thomas > > Am Donnerstag, 15. Juni 2017 schrieb Schäfer, Johannes : > >> Hi, >> >> Thanks for your investigation. I tried it today again with the last >> version from Github. For my test setup I still have the same numbers. So in >> average DeltaSpike needs 3 times longer that a pure EM implementation for >> finding entities by a query. >> For now I stopped my evaluation of DeltaSpike Data. My personal feeling >> is quite mixed. I really like to concept of DeltaSpike Data, but the >> performance loss is sometimes quite too big. >> I hope that there will be in future some improvements. >> >> Regards >> Johannes >> >> -----Original Message----- >> From: Thomas Andraschko [mailto:[email protected]] >> Sent: Friday, June 9, 2017 4:40 PM >> To: [email protected] >> Subject: Re: Performance of DeltaSpike Data >> >> FYI: >> >> i created a similar example based on the unittests in the ds data module. >> Here is the result of 100.000 selects, running on tomee and hsqldb: >> >> DS: 2600ms >> JPA: 1800ms >> >> >> 200-300ms are taken by our interceptor lookup, which can be improved a >> bit by caching the information about interceptors in the proxy instance >> instead of doing a bean+cache lookup each time. >> another 200-300ms are taken by DS Data directly - not sure if we can >> improve it further. >> >> Not sure about the other 200-400ms. >> >> 2017-06-08 16:33 GMT+02:00 Schäfer, Johannes <[email protected]>: >> >> > No chances with @ApplicationScoped. >> > To run my test just run "mvn clean install -PEmbeddedTestWeld". >> > It is using a in memory H2 DB. Maybe pipe the output into a log to >> > find the test results in the output. >> > >> > Grüße >> > Johannes >> > >> > >> > -----Original Message----- >> > From: Thomas Andraschko [mailto:[email protected]] >> > Sent: Thursday, June 8, 2017 3:12 PM >> > To: [email protected] >> > Subject: Re: Performance of DeltaSpike Data >> > >> > Could you also try to make your repository @ApplicationScoped? >> > >> > How can i run your tests? I don't like to install a AS or database ;) >> > >> > 2017-06-08 13:28 GMT+02:00 Schäfer, Johannes <[email protected]>: >> > >> > > Same with a locally compile version. >> > > Mysql: >> > > ____________________________________________________________ >> > > ____________________________________________________________ >> > > _____________________________ >> > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 | >> > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter >> > > 10240 | >> > > |=========================================================== >> > > ============================================================ >> > > =============================| >> > > | DS| 0.028479446| 0.088102335| 0.138260967| 0.255074252| >> > > | DS| 0.385907351| >> > > 0.734428279| 1.836123535| 4.125717222| 6.175937816| 13.217757392| >> > > 25.372525787| >> > > | EM| 0.010955534| 0.020851247| 0.041094277| 0.076565573| >> > > | EM| 0.195617863| >> > > 0.386509868| 0.812829151| 1.4044238 | 3.007676477| 6.232350452 | >> > > 11.726264467| >> > > >> > > Grüße >> > > Johannes >> > > >> > > >> > > -----Original Message----- >> > > From: Thomas Andraschko [mailto:[email protected]] >> > > Sent: Thursday, June 8, 2017 1:12 PM >> > > To: [email protected] >> > > Subject: Re: Performance of DeltaSpike Data >> > > >> > > please build DS from source, i don't think that SNAPSHOT is up to >> date. >> > > >> > > 2017-06-08 13:05 GMT+02:00 Schäfer, Johannes <[email protected]>: >> > > >> > > > Hi, >> > > > >> > > > Thanks for your great support. Now I had the time to run the tests. >> > > > Unfortunately no improvement. :-( >> > > > I used Mysql and H2 and both still have a significant difference. >> > > > mysql: >> > > > ____________________________________________________________ >> > > > ____________________________________________________________ >> > > > _____________________________ >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 >> | >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | >> iter >> > > > 10240 | >> > > > |=========================================================== >> > > > ============================================================ >> > > > =============================| >> > > > | DS| 0.042993818| 0.070756327| 0.139015158| 0.249963317| >> > > > | DS| 0.489673972| >> > > > 1.000932095| 1.418196146| 3.396942334| 6.268094687| 12.142304859| >> > > > 24.631240985| >> > > > | EM| 0.016741971| 0.034018415| 0.042539175| 0.097203944| >> > > > | EM| 0.15662194 >> > > > | EM| | >> > > > 0.32694476 | 0.665341891| 1.582051703| 2.602520533| 5.710082816 | >> > > > 10.856276852| >> > > > >> > > > h2: >> > > > ____________________________________________________________ >> > > > ____________________________________________________________ >> > > > __________________________ >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | iter 160 >> | >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | iter >> > > 10240 | >> > > > |=========================================================== >> > > > ============================================================ >> > > > ==========================| >> > > > | DS| 0.007259847| 0.009839833| 0.024182004| 0.040194493| >> > > > | DS| 0.037355253| >> > > > 0.038992501| 0.15695646| 0.157184542| 0.277937182| 0.575950893| >> > > > 0.848326297| >> > > > | EM| 7.12756E-4 | 7.15797E-4 | 0.0035079 | 0.001897262| >> > > > | EM| 0.003144109| >> > > > 0.007000594| 0.01269694| 0.024183904| 0.037443446| 0.108577248| >> > > > 0.217664259| >> > > > >> > > > I used version 1.8.1-SNAPSHOT for testing. >> > > > See https://github.com/johannesschaefer/javaee_jsf_ >> > > > cdi_jpa_data_ds_project_template >> > > > >> > > > Grüße >> > > > Johannes >> > > > >> > > > -----Original Message----- >> > > > From: Thomas Andraschko [mailto:[email protected]] >> > > > Sent: Thursday, June 8, 2017 10:36 AM >> > > > To: [email protected] >> > > > Subject: Re: Performance of DeltaSpike Data >> > > > >> > > > I did a major improvement and in my tests, both plain JPA and DS >> > > > Data are now very similar. >> > > > Would be great if you could provide the new numbers. >> > > > >> > > > 2017-06-07 14:33 GMT+02:00 Thomas Andraschko >> > > > <[email protected] >> > > > >: >> > > > >> > > > > Hi, >> > > > > >> > > > > could you please try to run your test again against the github >> > master? >> > > > > I already did a small improvement and refactored a little bit on >> > > > > the weekend. >> > > > > >> > > > > 2017-06-06 8:54 GMT+02:00 Schäfer, Johannes <[email protected]>: >> > > > > >> > > > >> Hi, >> > > > >> >> > > > >> So after the a long weekend, I'm back with my results. >> > > > >> For the write, findByPK and findAll tests I get now good numbers. >> > > > >> See: >> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ >> > > > >> ds_project_template/blob/master/src/test/java/de/psi/ >> > > > >> metals/futurelab/repo/benchmark/SaveTest.java >> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ >> > > > >> ds_project_template/blob/master/src/test/java/de/psi/ >> > > > >> metals/futurelab/repo/benchmark/ReadTest.java >> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ >> > > > >> ds_project_template/blob/master/src/test/java/de/psi/ >> > > > >> metals/futurelab/repo/benchmark/ReadAllTest.java >> > > > >> >> > > > >> The difference between delta spike and plain EM are just a few >> > > > >> percent, in both directions ;-) . >> > > > >> >> > > > >> But I wrote a new test case were I try to find entities by an >> query. >> > > > >> https://github.com/johannesschaefer/javaee_jsf_cdi_jpa_data_ >> > > > >> ds_project_template/blob/master/src/test/java/de/psi/ >> > > > >> metals/futurelab/repo/benchmark/ReadQueryTest.java >> > > > >> So I compare >> > > > >> TypedQuery< Material > query = eml.createQuery( >> > > > >> "SELECT m FROM Material m WHERE grade = :grade >> > > > >> AND width = :width AND thickness = :thickness", >> > > > >> Material.class ); >> > > > >> query.setParameter( "grade", "AAA" ); >> > > > >> query.setParameter( "width", 5 ); >> > > > >> query.setParameter( "thickness", 5. ); List< >> > > > >> Material >> > > > >> > mats = query.getResultList(); >> > > > >> >> > > > >> to >> > > > >> List< Material > mats = >> > > > >> matRepo.findByGradeAndWidthAndThickness( >> > > > >> "AAA", 5, 5. ); >> > > > >> >> > > > >> Here again the difference is quite high. >> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter >> 160 >> > | >> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | >> > iter >> > > > >> 10240 | >> > > > >> |=========================================================== >> > > > >> ============================================================ >> > > > >> =============================| >> > > > >> | DS| 0.03988012 | 0.151870613| 0.144881044| 0.270389952| >> > > > >> | DS| 0.526700787| >> > > > >> 1.023574545| 1.806960223| 3.426772405| 6.969935385| >> > > > >> 13.963582543| 26.785764953| >> > > > >> | EM| 0.010984804| 0.021940339| 0.059921297| 0.087386918| >> > > > >> | EM| 0.171045079| >> > > > >> 0.375059016| 0.747171594| 1.560946145| 2.968347174| 6.446844753 >> > > > >> | 12.361550486| >> > > > >> >> > > > >> So as you can see the DeltaSpike implementation needs at least >> > > > >> the double amount of time. >> > > > >> >> > > > >> Any hints for improving the performance? >> > > > >> >> > > > >> Regards, >> > > > >> Johannes >> > > > >> >> > > > >> -----Original Message----- >> > > > >> From: Schäfer, Johannes [mailto:[email protected]] >> > > > >> Sent: Thursday, June 1, 2017 2:27 PM >> > > > >> To: [email protected] >> > > > >> Subject: RE: Performance of DeltaSpike Data >> > > > >> >> > > > >> Right. Copy and paste error. >> > > > >> I added also a flush to the EM test. >> > > > >> Now I have similar numbers. >> > > > >> ____________________________________________________________ >> > > > >> ____________________________________________________________ >> > > > >> ______________________________ >> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter >> 160 >> > | >> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | >> > iter >> > > > >> 10240 | >> > > > >> |============================================================== >> > > > >> |== >> > > > >> |== >> > > > >> |== >> > > > >> |=== >> > > > >> |============================================================== >> > > > >> |== >> > > > >> |== >> > > > >> |== >> > > > >> |=== >> > > > >> |=======| >> > > > >> | DS| 0.001588214| 0.004130191| 0.007351854| 0.014062036| >> > > > >> | DS| 0.048373222| 0.593463008| 0.741351593| 1.697058004| >> > > > >> | DS| 6.049719288| 34.101109279| 101.589077365| >> > > > >> | EM| 0.001385601| 0.002662861| 0.004092937| 0.108730649| >> > > > >> | EM| 0.046299193| 0.106900289| 0.461147505| 1.688040769| >> > > > >> | EM| 5.960683928| 25.604583163| 106.688041149| >> > > > >> >> > > > >> It's a little bit strange for me, why I have to compare >> > > > >> EntityPersistenceRepository.save with a em.persist + em.flush. >> > > > >> I would expect that an simple EntityPersistenceRepository.save >> > > > >> don't have a flush (there is a separate >> EntityPersistenceRepository. >> > > > saveAndFlush). >> > > > >> >> > > > >> When I do a run with EntityPersistenceRepository.saveAndFlush I >> > > > >> get the following numbers. >> > > > >> ____________________________________________________________ >> > > > >> ____________________________________________________________ >> > > > >> ______________________________ >> > > > >> | | iter 10 | iter 20 | iter 40 | iter 80 | iter >> 160 >> > | >> > > > >> iter 320 | iter 640 | iter 1280 | iter 2560 | iter 5120 | >> > iter >> > > > >> 10240 | >> > > > >> |============================================================== >> > > > >> |== >> > > > >> |== >> > > > >> |== >> > > > >> |=== >> > > > >> |============================================================== >> > > > >> |== >> > > > >> |== >> > > > >> |== >> > > > >> |=== >> > > > >> |=======| >> > > > >> | DS| 0.001703015| 0.003457728| 0.008079817| 0.019099994| >> > > > >> | DS| 0.053865065| 0.940319597| 0.643245399| 2.292716685| >> > > > >> | DS| 9.902395587| 40.84301017 | 172.179435413| >> > > > >> | EM| 0.001677545| 0.004168205| 0.005779986| 0.014491211| >> > > > >> | EM| 0.031066334| 0.110747277| 0.4051742 | 1.925682412| >> > > > >> | EM| 5.842606084| 23.540393571| 132.817886521| >> > > > >> >> > > > >> So I have the feeling that there is still something wrong. >> > > > >> >> > > > >> Thanks to Gerhard for his additional hints. >> > > > >> I committed all changes to the github repo. >> > > > >> >> > > > >> Regards, >> > > > >> Johannes >> > > > >> >> > > > >> -----Original Message----- >> > > > >> From: Gerhard Petracek [mailto:[email protected]] >> > > > >> Sent: Thursday, June 1, 2017 1:21 PM >> > > > >> To: [email protected] >> > > > >> Subject: Re: Performance of DeltaSpike Data >> > > > >> >> > > > >> @johannes: >> > > > >> as mentioned yesterday you have to move EntityTransaction#begin >> > > > >> and EntityTransaction#commit into the for-loop. >> > > > >> >> > > > >> regards, >> > > > >> gerhard >> > > > >> >> > > > >> >> > > > >> >> > > > >> 2017-06-01 12:58 GMT+02:00 Thomas Andraschko >> > > > >> <[email protected] >> > > > >> >: >> > > > >> >> > > > >> > Hi, >> > > > >> > >> > > > >> > ~1 year ago i did many optimizations in the data module and >> > > > >> > AFAIR DS Data was only a little bit slower. >> > > > >> > After i compared my testcase with a benchmark from a user, >> > > > >> > the big different came from the transaction handling which >> > > > >> > was different in both testcases. >> > > > >> > >> > > > >> > Regards, >> > > > >> > Thomas >> > > > >> > >> > > > >> > 2017-06-01 12:33 GMT+02:00 Gerhard Petracek >> > > > >> > <[email protected] >> > >: >> > > > >> > >> > > > >> > > hi johannes, >> > > > >> > > >> > > > >> > > after refactoring your initial code to ds-test-control i >> > > > >> > > saw >> > e.g. >> > > > >> > > ~6s vs 7,5s for 2560 iterations. >> > > > >> > > i'll compare my local version with your new version >> > > > >> > > (mentioned in your mail). >> > > > >> > > >> > > > >> > > regards, >> > > > >> > > gerhard >> > > > >> > > >> > > > >> > > >> > > > >> > > >> > > > >> > > 2017-06-01 11:35 GMT+02:00 Schäfer, Johannes >> > > > >> > > <[email protected] >> > >: >> > > > >> > > >> > > > >> > > > Hi, >> > > > >> > > > >> > > > >> > > > My company is thinking about using DeltaSpike Data. But >> > > > >> > > > before we integrate this into our development I was asked >> > > > >> > > > to prepare some >> > > > >> > > benchmarks, >> > > > >> > > > comparing the usage of DeltaSpike Data with the usage of >> > > > >> > > > a plain EntityManager. >> > > > >> > > > I prepared some benchmarks and I was surprised that there >> > > > >> > > > is a big difference in the write performance. I already >> > > > >> > > > got some hints in the >> > > > >> > > delta >> > > > >> > > > spike irc channel, but the performance is still bad. >> > > > >> > > > Based on a template from os890 I implemented my tests and >> > > > >> > > > prepared a github project. >> > > > >> > > > https://github.com/johannesschaefer/javaee_jsf_ >> > > > >> > cdi_jpa_data_ds_project_ >> > > > >> > > > template >> > > > >> > > > Basically I'm talking about this test: >> > > > >> > > > https://github.com/johannesschaefer/javaee_jsf_ >> > > > >> > cdi_jpa_data_ds_project_ >> > > > >> > > > template/blob/master/src/test/java/de/psi/metals/futurela >> > > > >> > > > b/ repo/benchmark/SaveTest.java >> > > > >> > > > >> > > > >> > > > It just saves an entity into a DB in a loop. Depending of >> > > > >> > > > the number of iterations the difference is quite big. >> > > > >> > > > >> > > > >> > > > SaveTest >> > > > >> > > > _________________________________________________________ >> > > > >> > > > __ >> > > > >> > > > _ >> > > > >> > > > _________________________________________________________ >> > > > >> > > > __ _ _____________________________ >> > > > >> > > > | | iter 10 | iter 20 | iter 40 | iter 80 | >> iter >> > > 160 >> > > > >> | >> > > > >> > > > iter 320 | iter 640 | iter 1280 | iter 2560 | iter >> 5120 >> > > | >> > > > >> iter >> > > > >> > > > 10240 | >> > > > >> > > > |======================================================== >> > > > >> > > > |== >> > > > >> > > > |= >> > > > >> > > > ========================================================= >> > > > >> > > > == = =============================| >> > > > >> > > > | DS| 0.004911746| 0.03597043 | 0.015765787| 0.016966639| >> > > > >> > > > | DS| 0.043319612| >> > > > >> > > > 0.281807839| 1.308948835| 1.370535533| 8.186996818| >> > > > >> > > > 20.920141274| 93.631768475| >> > > > >> > > > | EM| 0.004557839| 0.003256631| 0.005775416| 0.004834958| >> > > > >> > > > | EM| 0.028243393| >> > > > >> > > > 0.035484616| 0.038600595| 0.088904458| 0.339158674| >> > > > >> > > > 0.745850523 >> > > > >> > > > | >> > > > >> > > > 0.853543234 | >> > > > >> > > > >> > > > >> > > > Also the difference between a run with 5120 and 10240 >> > > > >> > > > iteration is not just the double amount of time, it is >> > > > >> > > > more than 4 times >> > > > more. >> > > > >> > > > >> > > > >> > > > Do you have some hints to me what I'm doing wrong there? >> > > > >> > > > >> > > > >> > > > Regards >> > > > >> > > > Johannes >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > >> > > > >> > >> > > > >> >> > > > > >> > > > > >> > > > >> > > >> > >> >
