Excellent explanation! I don't know when I'll have enough time to do this but it sounds exciting. If anyone has the inclination please do it! It's an important fix.
On Mon, May 27, 2019 at 12:41 AM David Blevins <[email protected]> wrote: > > On May 26, 2019, at 5:26 AM, Richard Monson-Haefel < > [email protected]> wrote: > > > > This is really impressive, David! > > Thank you, Mentor :) > > > So, if I understand correctly, a new > > version of the bootstrap jar would load classes provided by Maven? Is > the > > time savings that Maven provides the uncompressed or is it something > else? > > Exactly right. > > Effectively, we take jars Maven has downloaded and put them into a Tomcat > zip and that becomes TomEE. When you write an Arquillian TomEE Remote test > case, we extract that zip each time the test runs. We boot the extracted > server, deploy your app, junit runs your tests in that test class, then we > delete the extracted server. Next test case the process repeats. > > On my mac it takes 4 seconds to extract TomEE Plus 8.0.0-M3, which is our > largest distro at 61MB. If you have 1000 tests, which is not a lot, you'll > spend 1 hour and 6 minutes just extracting TomEE zips. > > This is completely needless as the jars are already there downloaded by > Maven. We just need to use them. > > In the end the code that builds the Tomcat instance and would need to get > replaced with a Maven-aware version is not that big: > > mingus:/tmp 06:21:02 > $ unzip -l apache-tomee-webprofile-8.0.0-M3/bin/bootstrap.jar | grep > 'class$' > 8975 05-03-2019 23:26 > org/apache/catalina/security/SecurityClassLoad.class > 12196 05-03-2019 23:26 org/apache/catalina/startup/Bootstrap.class > 3494 05-03-2019 23:26 > org/apache/catalina/startup/CatalinaProperties.class > 1221 05-03-2019 23:26 > org/apache/catalina/startup/ClassLoaderFactory$1.class > 1207 05-03-2019 23:26 > org/apache/catalina/startup/ClassLoaderFactory$2.class > 1052 05-03-2019 23:26 > org/apache/catalina/startup/ClassLoaderFactory$Repository.class > 1389 05-03-2019 23:26 > org/apache/catalina/startup/ClassLoaderFactory$RepositoryType.class > 7429 05-03-2019 23:26 > org/apache/catalina/startup/ClassLoaderFactory.class > 858 05-03-2019 23:26 > > org/apache/catalina/startup/SafeForkJoinWorkerThreadFactory$SafeForkJoinWorkerThread.class > 934 05-03-2019 23:26 > org/apache/catalina/startup/SafeForkJoinWorkerThreadFactory.class > 5100 05-03-2019 23:26 org/apache/catalina/startup/Tool.class > 1414 05-03-2019 23:26 > org/apache/catalina/webresources/war/Handler.class > 1485 05-03-2019 23:26 > org/apache/catalina/webresources/war/WarURLConnection.class > 4387 05-03-2019 23:26 org/apache/tomcat/util/buf/UriUtil.class > > It's a 34k jar file with 14 classes in it. The Maven-aware version would > be slightly larger. > > The actual hard part will be eliminating the "build a TomEE zip" steps we > do in our build, such as TomEE webapps we discussed a few days back. > > > -David > > > > > > On Sat, May 25, 2019 at 8:34 PM David Blevins <[email protected]> > > wrote: > > > >> All, > >> > >> Our startup time in the TomEE 8.0.0 milestones is much slower. We've > >> historically had great startup speed, something we've worked on and it's > >> been a point of pride for the project. A core part of the "Be small, Be > >> certified, be Tomcat" slogan. > >> > >> We definitely need to put some work into this again. I created a very > >> rough prototype that allows us to benchmark TomEE versions against each > >> other and see if we are trending up or down. I ran it on all our > >> webprofile and plus versions from TomEE 1.0.0 to 8.0.0-M3 (build 1136). > >> > >> - > >> > https://issues.apache.org/jira/secure/attachment/12969800/startup-times.png > >> - > >> > https://issues.apache.org/jira/secure/attachment/12969802/sort-by-total.txt > >> - > >> > https://issues.apache.org/jira/secure/attachment/12969801/sort-by-startup.txt > >> > >> I dumped all the data into a JIRA so none of us have to search email for > >> it in 5 years time. > >> > >> - https://issues.apache.org/jira/browse/TOMEE-2528 > >> > >> Historical data is great, but the effort was put in with the hope we > could > >> potentially make some improvements. A few opportunities come to mind. > >> > >> # Performance Testing in the build > >> > >> The trick with automating performance tests is you need a reliable way > to > >> pass/fail without human interaction or you haven't achieved much. You > >> can't put in a fixed number like "must be faster than 5 seconds" as > >> everyone's hardware is different. Scientific method to the rescue. We > >> could introduce a control group and measure against that. The test > could > >> measure the speed of the current code + the previous two releases and > >> assert the current code is not slower than the previous two releases by > >> some margin. > >> > >> The test is then asserting simply, "we can't be slower than we were > >> before", which works on any hardware. > >> > >> This will likely cause intermittent build failures, but it will also > help > >> us squash performance issues early. They would be most likely to > happen on > >> library upgrades. This also means doing them the day before a release > >> would become much more of a no-no, which is perhaps not a bad thing. > >> > >> # Experimenting with Performance Improvements > >> > >> Tools like JMeter, Grinder, etc. don't help at all as they test > >> throughput. Tools like YourKit, JProfiler etc are great for seeing > where > >> time is spent in code and can be used for looking for optimizations. In > >> the end, however, they still take hours and it isn't always clear if > you're > >> moving forward. > >> > >> With this technique of a+b perf testing, we can race the previous code > and > >> our code with performance ideas and get bottom line feedback very > quickly. > >> You can test multiple builds so you can try a few ideas out at once and > see > >> which one moves the needle the most. > >> > >> If this tool is cleaned up enough and made very simple and command-line > >> friendly, I think it could turn us all into performance-minded > contributors. > >> > >> # Revisiting TomEE Embedded > >> > >> Already in the data, we can see 50% of our "new startup" time is > >> extracting the server from the tar.gz. We could speed things up by 50% > for > >> our build and everyone's build, just by skipping that step. > >> > >> We have always had a TomEE Embedded distribution, but we've approached > it > >> with a completely different mindset than regular TomEE and therefore it > has > >> less functionality and simply could not complete with a TomEE zip. > >> > >> I believe we took a wrong turn at the start. > >> > >> In a plain Tomcat zip the server is started with an incredibly small > >> classpath, basically just what is the bin/ dir. Tomcat will then load > >> everything in the lib/ dir into a new classloader, grab a class from > that > >> new classloader and tell it to finish the job. > >> > >> It would be possible for us to write an "embedded" version that does > >> exactly this, but using jars from the local Maven repo and would result > in > >> is being able to start stop many Tomcat/TomEE instances in one JVM. We > >> would have a slightly different version of the Tomcat bootstrap jar, but > >> everything after that point would be 100% the same. The embedded > version > >> would start 2x faster than a "remote" version but have the same > >> functionality and cost us very little in terms of maintenance. > >> > >> > >> > >> -- > >> David Blevins > >> http://twitter.com/dblevins > >> http://www.tomitribe.com > >> > >> > > > > -- > > Richard Monson-Haefel > > https://twitter.com/rmonson > > https://www.linkedin.com/in/monsonhaefel/ > > -- Richard Monson-Haefel https://twitter.com/rmonson https://www.linkedin.com/in/monsonhaefel/
