Hi Richard, Ok, when I clean the disk buffer cache (using echo 1 > /proc/sys/vm/drop_caches trick), it seems that when felix starts with a non-empty cache, it takes some substantial delay to load its existing cache before firing a "Framework Started" event.
(Our app server is installed by a specific bundle installer, which waits for the "Framework Started" event, before installing other bundles). I never noticed this delay because, generally, when I start my fwk in dev environment, the bundles are already available from linux buffer cache, and felix starts quickly. But if I do the "echo > ..." trick in order to clear disk cache, then the delay is important I mean: the framework takes time to load its existing cache. To be more specific, I describe exactly what I tested: - I install/start only one bundle (our bundle installer) - When started, the bundle installer listens to the "Framework Started" event, - So, when the start event is caught, the bundle installer starts installing/starting other bundles (190), which might be already present in the cache. Here are the logs of a cold start (disk buffers cleared, and felix cache empty): 2010-10-02 17:18:14,219 - Starting Felix OSGi Framework ... 2010-10-02 17:18:15,536 - BundleInstaller starting 2010-10-02 17:18:15,538 - Framework started: Installing 190 bundles ... 2010-10-02 17:18:20,562 - Done. And here are the logs of a cold start (disk buffers cleared, but existing felix cache with 190 bundles in it): 2010-10-02 17:18:58,167 - Starting Felix OSGi Framework ... 2010-10-02 17:19:07,705 - BundleInstaller starting 2010-10-02 17:19:07,707 - Framework started: Installing bundles ... 2010-10-02 17:19:07,709 - Done. -> Here, we see that when the cache is non empty, felix takes 9.6 seconds before starting the bundle installer: 2010-10-02 17:18:58,167 - Starting Felix OSGi Framework ... 2010-10-02 17:19:07,705 - BundleInstaller starting (at this point, the "Framework started" event is caught) What do you think ? /pierre On Fri, Oct 1, 2010 at 11:30 PM, <[email protected]> wrote: > On Fri, 1 Oct 2010 23:25:35 +0200, Pierre De Rop <[email protected]> > wrote: > > On Fri, Oct 1, 2010 at 10:57 PM, Richard S. Hall > > <[email protected]>wrote: > > > >> One thing, was your performance comparison based on using the new > cache > >> or > >> old? > >> > >> I assume you are using a newly created cache for the framework snapshot > >> and > >> [obviously] an old cache for the 3.0.2, correct? > >> > > yes: > > old cache with 3.0.2, and new cache with trunk. > > but I also checked that new fwk/trunk is properly working when being > > started > > with the old cache (from 3.0.2) > > > >> > >> You should also be testing an already created cache (i.e., a framework > >> restart), not an empty cache. > >> > > In the previous test, I compared with already created cache, not with > empty > > caches. > > With empty caches, here are the results: > > > > > > - old fwk 3.0.2 / fresh empty cache -> 15.8 sec. (this is surprising: > I > > would expect to get a longer delay, since the cache is empty when I > > start > > the fwk) > > > > 2010-10-01 23:10:17,403 Starting Felix 3.0.2 > > 2010-10-01 23:10:23,885 Starting cluster node "test" > > 2010-10-01 23:10:33,208 Cluster node "test" ready > > > > > > - new fwk / fresh empty cache ->15.2 sec > > > > 2010-10-01 23:15:46,2 Starting Felix 3.1.0.SNAPSHOT > > 2010-10-01 23:15:52,417 Starting cluster node "test" > > 2010-10-01 23:16:01,225 Cluster node "test" ready > > > > hmmm, I don't understand why the fwk startup time is better when the > cache > > is empty (15.2 sec) than when the cache is non-empty (21.2 sec from > > previous > > mail) > > Ok, I will do more tests ... may be I did something wrong ? ... to be > > continued ... > > Hmm. Yeah, that seems a little odd. In that case, you should just delete > your cache each time to get better startup performance! ;-) > > Definitely let me know what you find out. I too would expect framework > restarts to be faster since we don't have to copy the JAR files first. > > -> richard > > > > > /pierre > > > > > > > > > >> > >> Is that what you are doing? Just want to make sure I understand what is > >> being measured. :-) > >> > >> > >> -> richard > >> > >> On 10/1/10 16:43, Pierre De Rop wrote: > >> > >>> Hi Richard, > >>> > >>> I tried our app server with the trunk (old cache with the new > fwk/trunk) > >>> -> > >>> everything works ok > >>> (I install 195 bundles, and start 41 bundles) > >>> > >>> With new cache/fwk trunk -> it's also ok. > >>> > >>> I also compared the performance of the startup time between felix > 2.0.2 > >>> and > >>> felix trunk. > >>> To do this, I managed to ensure that my linux IO buffers cache are > empty > >>> before doing each test, by doing this: > >>> > >>> sync > >>> echo 1> /proc/sys/vm/drop_caches > >>> echo 2> /proc/sys/vm/drop_caches > >>> echo 3> /proc/sys/vm/drop_caches > >>> > >>> This ensures that my linux has released all buffer cache disk, like it > >>> is > >>> the case when I boot my host (cold start). > >>> Now, for felix 3.0.2, I load my server in around *21,1 *seconds (it's > >>> slow > >>> because all disk buffers are empty): > >>> > >>> 2010-10-01 22:15:07,880 Starting Felix 3.0.2 > >>> 2010-10-01 22:15:18,121 Starting cluster node "test" > >>> 2010-10-01 22:15:28,966 Cluster node "test" ready > >>> > >>> and with felix/trunk, indeed, the server starts a little bit quicker, > >>> but > >>> not so much: around *in 20,3* seconds (I also flushed disk buffers > >>> before > >>> starting the test): > >>> > >>> 2010-10-01 22:17:47,505 Starting Felix 3.1.0.SNAPSHOT > >>> 2010-10-01 22:17:56,815 Starting cluster node "test" > >>> 2010-10-01 22:18:07,798 Cluster node "test" ready > >>> > >>> > >>> /pierre > >>> > >>> > >>> On Fri, Oct 1, 2010 at 5:39 PM, Richard S. Hall<[email protected] > >>> >wrote: > >>> > >>> p.s. I deploy snapshots of the everything, including the convenience > >>>> framework distribution or you can build from trunk. > >>>> > >>>> > >>>> On 10/1/10 11:37, Richard S. Hall wrote: > >>>> > >>>> Hey everyone, > >>>>> > >>>>> I've made some changes to the bundle cache layout and handling to > >>>>> improve > >>>>> performance and clean up code. I've tried to maintain backward > >>>>> compatibility > >>>>> with old caches, but I'd appreciate is someone could try it out on > >>>>> their > >>>>> caches to see if it works. > >>>>> > >>>>> ** Back up your caches just in case! ** > >>>>> > >>>>> Note that the new cache format will not work with older frameworks. > So > >>>>> you > >>>>> cannot use a cache created with this new framework with previous > >>>>> frameworks; > >>>>> however, previously created caches should mostly continue to work > >>>>> between > >>>>> the two although there will be a loss of fidelity since changes to > >>>>> state > >>>>> are > >>>>> only saved to the new way or the old way depending on if you are > >>>>> running > >>>>> on > >>>>> the new framework or the old one. > >>>>> > >>>>> For the curious, I've combined five bundle cache files (id, > location, > >>>>> state, start level, last modified, and refresh count) into a single > >>>>> file > >>>>> and > >>>>> try to avoid excessive file accesses. This appears to improve > startup > >>>>> time > >>>>> when you have a cache with lots of bundles, but your mileage will > vary > >>>>> based > >>>>> on platform and other factors. Although, you won't necessarily see > any > >>>>> improvements if you are using an old cache, since it will revert to > >>>>> the > >>>>> old > >>>>> way. > >>>>> > >>>>> Thanks. > >>>>> > >>>>> -> richard > >>>>> > >>>>> >
