On 10/2/10 12:09, Pierre De Rop wrote:
Hi Richard,
Ok, when I clean the disk buffer cache (using echo 1>
/proc/sys/vm/drop_caches trick),
it seems that when felix starts with a non-empty cache, it takes some
substantial delay
to load its existing cache before firing a "Framework Started" event.
(Our app server is installed by a specific bundle installer, which waits for
the "Framework Started" event,
before installing other bundles).
I never noticed this delay because, generally, when I start my fwk in dev
environment, the bundles are already
available from linux buffer cache, and felix starts quickly.
But if I do the "echo> ..." trick in order to clear disk cache, then the
delay is important
I mean: the framework takes time to load its existing cache.
To be more specific, I describe exactly what I tested:
- I install/start only one bundle (our bundle installer)
- When started, the bundle installer listens to the "Framework Started"
event,
- So, when the start event is caught, the bundle installer starts
installing/starting
other bundles (190), which might be already present in the cache.
Here are the logs of a cold start (disk buffers cleared, and felix cache
empty):
2010-10-02 17:18:14,219 - Starting Felix OSGi Framework ...
2010-10-02 17:18:15,536 - BundleInstaller starting
2010-10-02 17:18:15,538 - Framework started: Installing 190 bundles ...
2010-10-02 17:18:20,562 - Done.
And here are the logs of a cold start (disk buffers cleared, but existing
felix cache with 190 bundles in it):
2010-10-02 17:18:58,167 - Starting Felix OSGi Framework ...
2010-10-02 17:19:07,705 - BundleInstaller starting
2010-10-02 17:19:07,707 - Framework started: Installing bundles ...
2010-10-02 17:19:07,709 - Done.
-> Here, we see that when the cache is non empty, felix takes 9.6 seconds
before starting the bundle installer:
It makes sense that it waits some time before firing the event, since
all previously installed bundles are reloaded BEFORE sending the event,
per the spec.
2010-10-02 17:18:58,167 - Starting Felix OSGi Framework ...
2010-10-02 17:19:07,705 - BundleInstaller starting (at this point, the
"Framework started" event is caught)
What do you think ?
Am I interpreting the above data correctly that it takes about 6 seconds
to start up when you install your bundles and 9.6 seconds for it to
start when the framework reloads the cached bundles? And this data is
for the trunk framework?
If so, that seems odd to me, since the install includes the copying of
the JAR file as well as opening the JAR file for use, while reloading
should only include opening the JAR file for use...
-> richard
/pierre
On Fri, Oct 1, 2010 at 11:30 PM,<[email protected]> wrote:
On Fri, 1 Oct 2010 23:25:35 +0200, Pierre De Rop<[email protected]>
wrote:
On Fri, Oct 1, 2010 at 10:57 PM, Richard S. Hall
<[email protected]>wrote:
One thing, was your performance comparison based on using the new
cache
or
old?
I assume you are using a newly created cache for the framework snapshot
and
[obviously] an old cache for the 3.0.2, correct?
yes:
old cache with 3.0.2, and new cache with trunk.
but I also checked that new fwk/trunk is properly working when being
started
with the old cache (from 3.0.2)
You should also be testing an already created cache (i.e., a framework
restart), not an empty cache.
In the previous test, I compared with already created cache, not with
empty
caches.
With empty caches, here are the results:
- old fwk 3.0.2 / fresh empty cache -> 15.8 sec. (this is surprising:
I
would expect to get a longer delay, since the cache is empty when I
start
the fwk)
2010-10-01 23:10:17,403 Starting Felix 3.0.2
2010-10-01 23:10:23,885 Starting cluster node "test"
2010-10-01 23:10:33,208 Cluster node "test" ready
- new fwk / fresh empty cache ->15.2 sec
2010-10-01 23:15:46,2 Starting Felix 3.1.0.SNAPSHOT
2010-10-01 23:15:52,417 Starting cluster node "test"
2010-10-01 23:16:01,225 Cluster node "test" ready
hmmm, I don't understand why the fwk startup time is better when the
cache
is empty (15.2 sec) than when the cache is non-empty (21.2 sec from
previous
mail)
Ok, I will do more tests ... may be I did something wrong ? ... to be
continued ...
Hmm. Yeah, that seems a little odd. In that case, you should just delete
your cache each time to get better startup performance! ;-)
Definitely let me know what you find out. I too would expect framework
restarts to be faster since we don't have to copy the JAR files first.
-> richard
/pierre
Is that what you are doing? Just want to make sure I understand what is
being measured. :-)
-> richard
On 10/1/10 16:43, Pierre De Rop wrote:
Hi Richard,
I tried our app server with the trunk (old cache with the new
fwk/trunk)
->
everything works ok
(I install 195 bundles, and start 41 bundles)
With new cache/fwk trunk -> it's also ok.
I also compared the performance of the startup time between felix
2.0.2
and
felix trunk.
To do this, I managed to ensure that my linux IO buffers cache are
empty
before doing each test, by doing this:
sync
echo 1> /proc/sys/vm/drop_caches
echo 2> /proc/sys/vm/drop_caches
echo 3> /proc/sys/vm/drop_caches
This ensures that my linux has released all buffer cache disk, like it
is
the case when I boot my host (cold start).
Now, for felix 3.0.2, I load my server in around *21,1 *seconds (it's
slow
because all disk buffers are empty):
2010-10-01 22:15:07,880 Starting Felix 3.0.2
2010-10-01 22:15:18,121 Starting cluster node "test"
2010-10-01 22:15:28,966 Cluster node "test" ready
and with felix/trunk, indeed, the server starts a little bit quicker,
but
not so much: around *in 20,3* seconds (I also flushed disk buffers
before
starting the test):
2010-10-01 22:17:47,505 Starting Felix 3.1.0.SNAPSHOT
2010-10-01 22:17:56,815 Starting cluster node "test"
2010-10-01 22:18:07,798 Cluster node "test" ready
/pierre
On Fri, Oct 1, 2010 at 5:39 PM, Richard S. Hall<[email protected]
wrote:
p.s. I deploy snapshots of the everything, including the convenience
framework distribution or you can build from trunk.
On 10/1/10 11:37, Richard S. Hall wrote:
Hey everyone,
I've made some changes to the bundle cache layout and handling to
improve
performance and clean up code. I've tried to maintain backward
compatibility
with old caches, but I'd appreciate is someone could try it out on
their
caches to see if it works.
** Back up your caches just in case! **
Note that the new cache format will not work with older frameworks.
So
you
cannot use a cache created with this new framework with previous
frameworks;
however, previously created caches should mostly continue to work
between
the two although there will be a loss of fidelity since changes to
state
are
only saved to the new way or the old way depending on if you are
running
on
the new framework or the old one.
For the curious, I've combined five bundle cache files (id,
location,
state, start level, last modified, and refresh count) into a single
file
and
try to avoid excessive file accesses. This appears to improve
startup
time
when you have a cache with lots of bundles, but your mileage will
vary
based
on platform and other factors. Although, you won't necessarily see
any
improvements if you are using an old cache, since it will revert to
the
old
way.
Thanks.
-> richard
