sszuev commented on issue #1961: URL: https://github.com/apache/jena/issues/1961#issuecomment-1893205295
The tests are based on various scenarios that are executed in multi threading and in cycles. There are `CyclicBarrier` and `CountDownLatch` that provide simultaneous execution of scenarios. There are also checks for invariants. This is a classic scheme, without barriers we can't be sure that there is really multithreading execution. Scenarios are different, some only R, some W, most are RW. These scenarios were written without any particular understanding of what needs to be tested, i.e. this is a set of some operations that were collected into one scenario based on some intuitive thoughts. Benchmarks also contains functional, i.e. classic, benchmarks, where single operation is measured. --------------- I added your implementation to benchmarks and tests suites. see branch jena-5.x.x-SWMR; https://github.com/sszuev/concurrent-rdf-graph/tree/jena-5.x.x-SWMR Note that I changed version of project `5.0.0-SNAPSHOT-SWMR` in order to work with the official `5.0.0-SNAPSHOT` and your branch installed locally without re-installing. --------------- I run benchmarks with the parameters `-wi 2 -i 3`, Total time: 08:00:12 A classic SYNCHRONIZED_GRAPH_V1 is the faster one, WRAPPER_TRANSACTIONAL2_GRAPH on the last place. And important note: there are failures when running WRAPPER_TRANSACTIONAL2_GRAPH, so not all benchmarks have been collected. See logs attached. ``` Benchmark (factory) Mode Cnt Score Error Units BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_6x21 TXN_GRAPH thrpt 15 64,930 ± 7,678 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_6x21 SYNCHRONIZED_GRAPH_V1 thrpt 15 232,215 ± 38,918 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_6x21 SYNCHRONIZED_GRAPH_V2 thrpt 15 241,507 ± 26,096 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_6x21 RW_LOCKING_GRAPH_V1 thrpt 15 231,456 ± 49,945 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_6x21 RW_LOCKING_GRAPH_V2 thrpt 15 232,684 ± 34,847 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_6x21 WRAPPER_TRANSACTIONAL2_GRAPH thrpt 15 0,069 ± 0,001 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_F_RW_5x4 TXN_GRAPH thrpt 15 6,328 ± 0,291 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_F_RW_5x4 SYNCHRONIZED_GRAPH_V1 thrpt 15 4,606 ± 1,331 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_F_RW_5x4 SYNCHRONIZED_GRAPH_V2 thrpt 15 2,954 ± 0,220 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_F_RW_5x4 RW_LOCKING_GRAPH_V1 thrpt 15 4,845 ± 0,465 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_F_RW_5x4 RW_LOCKING_GRAPH_V2 thrpt 15 4,765 ± 0,541 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_F_RW_5x4 WRAPPER_TRANSACTIONAL2_GRAPH thrpt 15 0,067 ± 0,001 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_G_R_4x11 TXN_GRAPH thrpt 15 1,131 ± 0,016 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_G_R_4x11 SYNCHRONIZED_GRAPH_V1 thrpt 15 159,767 ± 17,675 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_G_R_4x11 SYNCHRONIZED_GRAPH_V2 thrpt 15 156,280 ± 16,911 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_G_R_4x11 RW_LOCKING_GRAPH_V1 thrpt 15 161,223 ± 26,616 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_G_R_4x11 RW_LOCKING_GRAPH_V2 thrpt 15 158,052 ± 20,801 ops/s BigGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_G_R_4x11 WRAPPER_TRANSACTIONAL2_GRAPH thrpt 15 174,225 ± 40,285 ops/s FunctionalBenchmarks.ADD SYNCHRONIZED_GRAPH_V1 thrpt 15 9507833,994 ± 869229,620 ops/s FunctionalBenchmarks.ADD SYNCHRONIZED_GRAPH_V2 thrpt 15 9364786,667 ± 375206,535 ops/s FunctionalBenchmarks.ADD RW_LOCKING_GRAPH_V1 thrpt 15 9484691,317 ± 181337,215 ops/s FunctionalBenchmarks.ADD RW_LOCKING_GRAPH_V2 thrpt 15 9560358,404 ± 215175,600 ops/s FunctionalBenchmarks.ADD TXN_GRAPH thrpt 15 578039,756 ± 19580,417 ops/s FunctionalBenchmarks.ADD MEM_GRAPH thrpt 15 10340640,978 ± 728912,121 ops/s FunctionalBenchmarks.ADD WRAPPER_TRANSACTIONAL2_GRAPH thrpt 15 3252,095 ± 781,614 ops/s FunctionalBenchmarks.CONTAINS SYNCHRONIZED_GRAPH_V1 thrpt 15 13899056,559 ± 739607,369 ops/s FunctionalBenchmarks.CONTAINS SYNCHRONIZED_GRAPH_V2 thrpt 15 13335406,849 ± 775383,488 ops/s FunctionalBenchmarks.CONTAINS RW_LOCKING_GRAPH_V1 thrpt 15 12737718,035 ± 264032,537 ops/s FunctionalBenchmarks.CONTAINS RW_LOCKING_GRAPH_V2 thrpt 15 12967674,455 ± 733900,276 ops/s FunctionalBenchmarks.CONTAINS TXN_GRAPH thrpt 15 1011517,407 ± 35875,156 ops/s FunctionalBenchmarks.CONTAINS MEM_GRAPH thrpt 15 14939271,773 ± 139467,875 ops/s FunctionalBenchmarks.CONTAINS WRAPPER_TRANSACTIONAL2_GRAPH thrpt 15 3733,116 ± 1102,551 ops/s FunctionalBenchmarks.COUNT SYNCHRONIZED_GRAPH_V1 thrpt 15 20713402,946 ± 495506,189 ops/s FunctionalBenchmarks.COUNT SYNCHRONIZED_GRAPH_V2 thrpt 15 20743633,609 ± 276745,250 ops/s FunctionalBenchmarks.COUNT RW_LOCKING_GRAPH_V1 thrpt 15 18677039,197 ± 422703,739 ops/s FunctionalBenchmarks.COUNT RW_LOCKING_GRAPH_V2 thrpt 15 18545840,762 ± 583214,365 ops/s FunctionalBenchmarks.COUNT TXN_GRAPH thrpt 15 113679,704 ± 2759,275 ops/s FunctionalBenchmarks.COUNT MEM_GRAPH thrpt 15 22498971,119 ± 182178,216 ops/s FunctionalBenchmarks.DELETE SYNCHRONIZED_GRAPH_V1 thrpt 15 6531783,863 ± 111434,950 ops/s FunctionalBenchmarks.DELETE SYNCHRONIZED_GRAPH_V2 thrpt 15 5980168,481 ± 138766,198 ops/s FunctionalBenchmarks.DELETE RW_LOCKING_GRAPH_V1 thrpt 15 6004025,859 ± 188190,779 ops/s FunctionalBenchmarks.DELETE RW_LOCKING_GRAPH_V2 thrpt 15 5828131,807 ± 71189,885 ops/s FunctionalBenchmarks.DELETE TXN_GRAPH thrpt 15 457443,775 ± 2200,891 ops/s FunctionalBenchmarks.DELETE MEM_GRAPH thrpt 15 6794662,701 ± 141559,073 ops/s FunctionalBenchmarks.DELETE WRAPPER_TRANSACTIONAL2_GRAPH thrpt 15 3492,775 ± 628,057 ops/s FunctionalBenchmarks.FIND_ALL SYNCHRONIZED_GRAPH_V1 thrpt 15 2436745,158 ± 47707,699 ops/s FunctionalBenchmarks.FIND_ALL SYNCHRONIZED_GRAPH_V2 thrpt 15 785148,924 ± 12140,536 ops/s FunctionalBenchmarks.FIND_ALL RW_LOCKING_GRAPH_V1 thrpt 15 777309,448 ± 8278,041 ops/s FunctionalBenchmarks.FIND_ALL RW_LOCKING_GRAPH_V2 thrpt 15 780578,613 ± 6091,941 ops/s FunctionalBenchmarks.FIND_ALL TXN_GRAPH thrpt 15 113831,962 ± 5882,211 ops/s FunctionalBenchmarks.FIND_ALL MEM_GRAPH thrpt 15 4071771,498 ± 48533,407 ops/s FunctionalBenchmarks.FIND_ALL WRAPPER_TRANSACTIONAL2_GRAPH thrpt 3 4205,814 ± 12821,396 ops/s FunctionalBenchmarks.FIND_SOME SYNCHRONIZED_GRAPH_V1 thrpt 15 9878544,973 ± 130398,809 ops/s FunctionalBenchmarks.FIND_SOME SYNCHRONIZED_GRAPH_V2 thrpt 15 1895768,662 ± 21455,858 ops/s FunctionalBenchmarks.FIND_SOME RW_LOCKING_GRAPH_V1 thrpt 15 1868735,058 ± 22717,077 ops/s FunctionalBenchmarks.FIND_SOME RW_LOCKING_GRAPH_V2 thrpt 15 1864042,205 ± 33819,169 ops/s FunctionalBenchmarks.FIND_SOME TXN_GRAPH thrpt 15 821360,951 ± 17775,343 ops/s FunctionalBenchmarks.FIND_SOME MEM_GRAPH thrpt 15 16829918,234 ± 190392,137 ops/s FunctionalBenchmarks.FIND_SOME WRAPPER_TRANSACTIONAL2_GRAPH thrpt 3 2870,380 ± 10246,865 ops/s FunctionalBenchmarks.MIXED_OPERATIONS SYNCHRONIZED_GRAPH_V1 thrpt 15 429112,584 ± 5577,188 ops/s FunctionalBenchmarks.MIXED_OPERATIONS SYNCHRONIZED_GRAPH_V2 thrpt 15 166994,601 ± 2825,367 ops/s FunctionalBenchmarks.MIXED_OPERATIONS RW_LOCKING_GRAPH_V1 thrpt 15 164384,369 ± 2256,327 ops/s FunctionalBenchmarks.MIXED_OPERATIONS RW_LOCKING_GRAPH_V2 thrpt 15 163507,147 ± 1040,422 ops/s FunctionalBenchmarks.MIXED_OPERATIONS TXN_GRAPH thrpt 15 39756,133 ± 1235,483 ops/s FunctionalBenchmarks.MIXED_OPERATIONS MEM_GRAPH thrpt 15 502636,605 ± 11395,114 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_A_RW_5x6 TXN_GRAPH thrpt 15 173,808 ± 6,032 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_A_RW_5x6 SYNCHRONIZED_GRAPH_V1 thrpt 15 416,793 ± 4,074 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_A_RW_5x6 SYNCHRONIZED_GRAPH_V2 thrpt 15 196,966 ± 2,568 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_A_RW_5x6 RW_LOCKING_GRAPH_V1 thrpt 15 176,334 ± 1,397 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_A_RW_5x6 RW_LOCKING_GRAPH_V2 thrpt 15 179,059 ± 6,782 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_B_R_5x5 TXN_GRAPH thrpt 15 216,288 ± 5,114 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_B_R_5x5 SYNCHRONIZED_GRAPH_V1 thrpt 15 278,412 ± 3,210 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_B_R_5x5 SYNCHRONIZED_GRAPH_V2 thrpt 15 219,668 ± 1,491 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_B_R_5x5 RW_LOCKING_GRAPH_V1 thrpt 15 266,005 ± 3,070 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_B_R_5x5 RW_LOCKING_GRAPH_V2 thrpt 15 267,921 ± 2,225 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_B_R_5x5 WRAPPER_TRANSACTIONAL2_GRAPH thrpt 6 345,026 ± 98,806 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_C_RW_8x14 TXN_GRAPH thrpt 15 83,603 ± 0,789 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_C_RW_8x14 SYNCHRONIZED_GRAPH_V1 thrpt 15 422,064 ± 10,011 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_C_RW_8x14 SYNCHRONIZED_GRAPH_V2 thrpt 15 123,621 ± 1,448 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_C_RW_8x14 RW_LOCKING_GRAPH_V1 thrpt 15 83,957 ± 1,430 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_C_RW_8x14 RW_LOCKING_GRAPH_V2 thrpt 15 82,960 ± 1,118 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_C_RW_8x14 WRAPPER_TRANSACTIONAL2_GRAPH thrpt 15 5,725 ± 0,117 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_DC_RW_6x14 TXN_GRAPH thrpt 15 47,662 ± 1,784 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_DC_RW_6x14 SYNCHRONIZED_GRAPH_V1 thrpt 15 273,537 ± 8,762 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_DC_RW_6x14 SYNCHRONIZED_GRAPH_V2 thrpt 15 88,320 ± 1,513 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_DC_RW_6x14 RW_LOCKING_GRAPH_V1 thrpt 15 58,757 ± 0,697 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_DC_RW_6x14 RW_LOCKING_GRAPH_V2 thrpt 15 58,869 ± 0,793 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_DC_RW_6x14 WRAPPER_TRANSACTIONAL2_GRAPH thrpt 12 4,383 ± 0,085 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_5x5 TXN_GRAPH thrpt 15 301,122 ± 27,625 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_5x5 SYNCHRONIZED_GRAPH_V1 thrpt 15 1217,857 ± 11,667 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_5x5 SYNCHRONIZED_GRAPH_V2 thrpt 15 1190,543 ± 15,668 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_5x5 RW_LOCKING_GRAPH_V1 thrpt 15 1202,480 ± 20,628 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_5x5 RW_LOCKING_GRAPH_V2 thrpt 15 1201,385 ± 16,531 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_D_W_5x5 WRAPPER_TRANSACTIONAL2_GRAPH thrpt 15 200,902 ± 7,643 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_H_RW_4x8 TXN_GRAPH thrpt 15 140,139 ± 3,358 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_H_RW_4x8 SYNCHRONIZED_GRAPH_V1 thrpt 15 215,947 ± 2,394 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_H_RW_4x8 SYNCHRONIZED_GRAPH_V2 thrpt 15 126,584 ± 1,533 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_H_RW_4x8 RW_LOCKING_GRAPH_V1 thrpt 15 126,836 ± 1,002 ops/s PizzaGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_H_RW_4x8 RW_LOCKING_GRAPH_V2 thrpt 15 127,282 ± 1,476 ops/s SmallGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_K_RW_5x5 TXN_GRAPH thrpt 15 546,588 ± 4,167 ops/s SmallGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_K_RW_5x5 SYNCHRONIZED_GRAPH_V1 thrpt 15 2224,895 ± 30,334 ops/s SmallGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_K_RW_5x5 SYNCHRONIZED_GRAPH_V2 thrpt 15 1599,416 ± 14,426 ops/s SmallGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_K_RW_5x5 RW_LOCKING_GRAPH_V1 thrpt 15 1384,214 ± 31,430 ops/s SmallGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_K_RW_5x5 RW_LOCKING_GRAPH_V2 thrpt 15 1380,612 ± 22,594 ops/s SmallGraphConcurrentBenchmarks.CONCURRENT_SCENARIO_K_RW_5x5 WRAPPER_TRANSACTIONAL2_GRAPH thrpt 8 61,845 ± 6,815 ops/s ``` [benchmarks-160124-01.txt](https://github.com/apache/jena/files/13946789/benchmarks-160124-01.txt) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
