[
https://issues.apache.org/jira/browse/TOMEE-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Zowalla closed TOMEE-2528.
----------------------------------
Resolution: Won't Fix
Closing. This records benchmark data for versions that are end-of-life; the
data stays here for reference.
> Research: TomEE startup times v1.0.0 to 8.0.0-M3
> ------------------------------------------------
>
> Key: TOMEE-2528
> URL: https://issues.apache.org/jira/browse/TOMEE-2528
> Project: TomEE
> Issue Type: Documentation
> Affects Versions: 1.7.0, 7.0.0, 7.1, 8.0.0-M1
> Reporter: David Blevins
> Assignee: David Blevins
> Priority: Major
> Attachments: sort-by-startup.txt, sort-by-total.txt, speed.war,
> startup-times.png
>
>
> The following are startup times and execution of a simple JAX-RS application
> on TomEE 1.0.0 to 8.0.0-M3 (still up for vote). Using a JIRA to collect the
> research as it will be very hard to find in a few years if only in email.
> What is being measured is the time to extract the tar.gz binary, copy in a
> simple war with a JAX-RS service, boot the server and execute the service
> once. This is the round trip paid on each Arquillian test case that uses the
> TomEE Remote Arquillian Adapter.
> !startup-times.png|height=315,width=567!
> The columns are:
> - binary name
> - number of times started
> - milliseconds to extract (milliseconds, 90th percentile)
> - milliseconds to start (90th percentile)
> - total milliseconds of extract, start, first invoke (90th percentile)
> - visual bar (blue x is extract time, red s is startup time)
> h2. Sort by Total Time
> Here is the raw data sorted by total time:
> {noformat}
> apache-tomee-1.5.2-jaxrs 140 1651 2245 3867
> xxxxxxxxxxxxxxxxxsssssssssssssssssssss
> apache-tomee-1.7.2-jaxrs 140 2229 2182 4457
> xxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssss
> apache-tomee-1.6.0-jaxrs 140 2166 2363 4457
> xxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-1.7.3-jaxrs 139 2297 2203 4497
> xxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssss
> apache-tomee-1.5.1-jaxrs 140 2075 2479 4503
> xxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> apache-tomee-1.7.4-jaxrs 140 2357 2138 4503
> xxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssss
> apache-tomee-1.5.0-jaxrs 140 2030 2518 4521
> xxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> apache-tomee-7.0.0-M1-webprofile 140 2252 2206 4526
> xxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-7.0.0-webprofile 140 2285 2192 4542
> xxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-7.0.1-webprofile 139 2289 2193 4547
> xxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-1.7.5-jaxrs 139 2274 2283 4565
> xxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-7.0.0-M3-webprofile 139 2264 2198 4595
> xxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-7.0.2-webprofile 139 2343 2197 4620
> xxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-7.0.3-webprofile 139 2372 2190 4651
> xxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-7.0.0-M2-webprofile 138 2375 2208 4672
> xxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-1.7.0-jaxrs 139 2345 2452 4717
> xxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssss
> apache-tomee-7.0.5-webprofile 139 2449 2243 4755
> xxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-1.7.1-jaxrs 139 2338 2387 4759
> xxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssss
> apache-tomee-7.0.4-webprofile 139 2445 2223 4759
> xxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-1.6.0.1-jaxrs 139 2136 2739 4840
> xxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssss
> apache-tomee-7.1.0-webprofile 139 2545 2240 4859
> xxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-1.6.0.2-jaxrs 139 2353 2772 5069
> xxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssss
> apache-tomee-1.5.2-plus 138 3046 2425 5409
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssss
> apache-tomee-1.0.0-plus 138 2952 2433 5417
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> apache-tomee-1.7.3-plus 138 3213 2340 5537
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-8.0.0-M2-webprofile 139 2414 3156 5543
> xxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssss
> apache-tomee-1.7.2-plus 139 3226 2325 5551
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> apache-tomee-8.0.0-M1-webprofile 139 2420 3131 5593
> xxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssss
> apache-tomee-1.7.4-plus 139 3317 2266 5595
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssss
> apache-tomee-8.0.0-M3-webprofile 139 2564 3097 5636
> xxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssss
> apache-tomee-7.0.1-plus 139 3250 2431 5717
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> apache-tomee-1.5.1-plus 139 3109 2682 5723
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssssss
> apache-tomee-1.7.0-plus 138 3278 2522 5730
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> apache-tomee-1.7.5-plus 139 3344 2415 5732
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssss
> apache-tomee-1.7.1-plus 139 3314 2485 5748
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssss
> apache-tomee-1.5.0-plus 139 3106 2762 5793
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssssss
> apache-tomee-7.0.0-M2-plus 139 3314 2475 5827
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> apache-tomee-7.0.0-plus 138 3339 2424 5874
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> apache-tomee-7.0.2-plus 139 3370 2450 5919
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssssss
> apache-tomee-7.0.3-plus 139 3434 2429 5929
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> apache-tomee-1.6.0-plus 139 3420 2544 5938
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> apache-tomee-7.0.0-M3-plus 139 3419 2451 5944
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> apache-tomee-7.0.0-M1-plus 139 3555 2422 6034
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> apache-tomee-7.0.4-plus 139 3510 2466 6111
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssssss
> apache-tomee-7.0.5-plus 139 3691 2526 6263
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssssss
> apache-tomee-7.1.0-plus 139 3684 2526 6332
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssss
> apache-tomee-1.6.0.1-plus 139 3472 2942 6335
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssss
> apache-tomee-1.6.0.2-plus 139 3556 2956 6451
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssss
> apache-tomee-8.0.0-M2-plus 138 3579 3434 7087
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssssssss
> apache-tomee-8.0.0-M1-plus 139 3670 3435 7136
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssssssss
> apache-tomee-8.0.0-M3-plus 139 4002 3493 7513
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssssssss
> {noformat}
> h2. Sort by Startup Time
> Here is the raw data sorted by just the start time, excluding time to extract
> the binary.
> {noformat}
> 2138 apache-tomee-1.7.4-jaxrs 140 2357 4503
> xxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssss
> 2182 apache-tomee-1.7.2-jaxrs 140 2229 4457
> xxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssss
> 2190 apache-tomee-7.0.3-webprofile 139 2372 4651
> xxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2192 apache-tomee-7.0.0-webprofile 140 2285 4542
> xxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2193 apache-tomee-7.0.1-webprofile 139 2289 4547
> xxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2197 apache-tomee-7.0.2-webprofile 139 2343 4620
> xxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2198 apache-tomee-7.0.0-M3-webprofile 139 2264 4595
> xxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2203 apache-tomee-1.7.3-jaxrs 139 2297 4497
> xxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssss
> 2206 apache-tomee-7.0.0-M1-webprofile 140 2252 4526
> xxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2208 apache-tomee-7.0.0-M2-webprofile 138 2375 4672
> xxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2223 apache-tomee-7.0.4-webprofile 139 2445 4759
> xxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2240 apache-tomee-7.1.0-webprofile 139 2545 4859
> xxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2243 apache-tomee-7.0.5-webprofile 139 2449 4755
> xxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2245 apache-tomee-1.5.2-jaxrs 140 1651 3867
> xxxxxxxxxxxxxxxxxsssssssssssssssssssss
> 2266 apache-tomee-1.7.4-plus 139 3317 5595
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssss
> 2283 apache-tomee-1.7.5-jaxrs 139 2274 4565
> xxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2325 apache-tomee-1.7.2-plus 139 3226 5551
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2340 apache-tomee-1.7.3-plus 138 3213 5537
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2363 apache-tomee-1.6.0-jaxrs 140 2166 4457
> xxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssss
> 2387 apache-tomee-1.7.1-jaxrs 139 2338 4759
> xxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssss
> 2415 apache-tomee-1.7.5-plus 139 3344 5732
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssss
> 2422 apache-tomee-7.0.0-M1-plus 139 3555 6034
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> 2424 apache-tomee-7.0.0-plus 138 3339 5874
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> 2425 apache-tomee-1.5.2-plus 138 3046 5409
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssss
> 2429 apache-tomee-7.0.3-plus 139 3434 5929
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> 2431 apache-tomee-7.0.1-plus 139 3250 5717
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> 2433 apache-tomee-1.0.0-plus 138 2952 5417
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> 2450 apache-tomee-7.0.2-plus 139 3370 5919
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssssss
> 2451 apache-tomee-7.0.0-M3-plus 139 3419 5944
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> 2452 apache-tomee-1.7.0-jaxrs 139 2345 4717
> xxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssss
> 2466 apache-tomee-7.0.4-plus 139 3510 6111
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssssss
> 2475 apache-tomee-7.0.0-M2-plus 139 3314 5827
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> 2479 apache-tomee-1.5.1-jaxrs 140 2075 4503
> xxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> 2485 apache-tomee-1.7.1-plus 139 3314 5748
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssss
> 2518 apache-tomee-1.5.0-jaxrs 140 2030 4521
> xxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> 2522 apache-tomee-1.7.0-plus 138 3278 5730
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> 2526 apache-tomee-7.0.5-plus 139 3691 6263
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssssss
> 2526 apache-tomee-7.1.0-plus 139 3684 6332
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssss
> 2544 apache-tomee-1.6.0-plus 139 3420 5938
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssss
> 2682 apache-tomee-1.5.1-plus 139 3109 5723
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssssss
> 2739 apache-tomee-1.6.0.1-jaxrs 139 2136 4840
> xxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssss
> 2762 apache-tomee-1.5.0-plus 139 3106 5793
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsssssssssssssssssssssssss
> 2772 apache-tomee-1.6.0.2-jaxrs 139 2353 5069
> xxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssss
> 2942 apache-tomee-1.6.0.1-plus 139 3472 6335
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssss
> 2956 apache-tomee-1.6.0.2-plus 139 3556 6451
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssss
> 3097 apache-tomee-8.0.0-M3-webprofile 139 2564 5636
> xxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssss
> 3131 apache-tomee-8.0.0-M1-webprofile 139 2420 5593
> xxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssss
> 3156 apache-tomee-8.0.0-M2-webprofile 139 2414 5543
> xxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssss
> 3434 apache-tomee-8.0.0-M2-plus 138 3579 7087
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssssssss
> 3435 apache-tomee-8.0.0-M1-plus 139 3670 7136
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssssssss
> 3493 apache-tomee-8.0.0-M3-plus 139 4002 7513
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxssssssssssssssssssssssssssssssssss
> {noformat}
> h2. Machine Info
> {noformat}
> Model Name: MacBook Pro
> Model Identifier: MacBookPro11,5
> Processor Name: Intel Core i7
> Processor Speed: 2.8 GHz
> Number of Processors: 1
> Total Number of Cores: 4
> L2 Cache (per Core): 256 KB
> L3 Cache: 6 MB
> Memory: 16 GB
> Boot ROM Version: MBP114.0172.B25
> SMC Version (system): 2.30f2
> {noformat}
> Test time was roughly 10 hours.
> h2. Test Harness
> Tests were done with this exact revision of this test case:
> -
> https://github.com/apache/tomee/blob/c232f01d959d815a518b11221d793bdf2cc6d605/itests/startup/src/test/java/org/apache/tomee/itest/startup/StartupTest.java#L148
> And the attached simple JAX-RS application compiled separately using Java 6
> so it could run on all versions of TomEE.
> The test case is very crude and contains hardcoded and non-portable file
> paths. It is really a prototype at this point. More cleanup and work would
> be required for it to actually stay on the build. It was checked in so that
> even if we remove it, it is still in our history and we can come back and
> revive and finish it at any point.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
