I am timing the time it takes for KernelBootstrap.initialize() to complete, just like in Main. The times I gave are without YourKit profiling, sorry that was unclear. When I run the initialization with YourKit its more like 13-14 seconds.
I moved all of the ScalaInterpreter creation and initialization earlier, which I think is most of the work? It includes all the bindVariables stuff. Thanks for the input. It sounds like there's some interest so I'll start working on this change. Thanks Brad > On Jan 4, 2018, at 4:26 PM, Luciano Resende <[email protected]> wrote: > > On Thu, Jan 4, 2018 at 1:09 PM, Bradley Kaiser <[email protected]> > wrote: > >> Hey Guys, >> >> I've been looking at ways to speed up Toree's startup and came up with a >> quick proof of concept that took the startup time from around 6.5 seconds >> to 5.5 seconds on my laptop. I'd like to check committer interest in this >> before I go any further. This change will require rearranging the >> KernelBootstrap class. >> >> > What are you considering for the startup time ? To me, measuring it via > yourkit as well, the Main.scala was consuming 13 secs. > > > >> My proof of concept is simple, I saw that the slowest part of startup was >> starting the Scala interpreter, so I ripped that out, moved it to its own >> thread, and started it very early in the bootstrap process. This way while >> it runs the rest of the kernel bootstrap can continue. I did this in a >> hacky way just to see if it would work. It would need to be implemented for >> real. >> >> I tested the startup time by setting up a test harness that just runs the >> kernel bootstrap and times it. I used YourKit to profile the application >> and see what was taking the most time. >> >> Here are the results from my tests. Time is in seconds. >> >> run original changed >> 1 6.876 5.2 >> 2 6.445 5.8 >> 3 6.527 5.7 >> 4 6.676 5.4 >> avg 6.631 5.525 >> > > Interesting, as my measurings was telling me that some big portions of the > time was being spent on performing initialization tasks which required the > scala interpreter to be ready. Consolidating the multiple calls with some > other minor tweaks got about 2 secs of performance improvement on the > kernel initialization. > > Another area of possible improvement might be configuring a flag to disable > auto-loading of all different plugins, which would save us another 1 sec > (e.g. not loading other interpreter plugins which are not used when toree > is mainly used as a scala kernel) > > >> >> If you guys think this is a reasonable approach let me know and i will >> make a ticket and work on implementing it for real. I think this speedup >> will work nicely with the changes proposed in TOREE-444 also. >> >> > I have attached my initial findings at > https://issues.apache.org/jira/browse/TOREE-444 > > > Others, any possible additions here ? > > -- > Luciano Resende > http://twitter.com/lresende1975 > http://lresende.blogspot.com/
