Sylvain Wallez wrote:
>
>>Ok I will quote Berin as answer:
>>"...That way during extreme load conditions the number of times we call
>>the "lastModified" method doesn't change. Instead of 1/request
>>(with 200 simultaneous users requesting 4 pages a second that comes to
>>800 calls a second) it is once per period of time. Even at one second,
>>you have called "lastModified" 1/800th of the time using the afforementioned
>>example. It defaults to once per minute which is 1/24000th of the time (that
>>is 2400000% decrease in calls..."
>>
>>IMHO that sound reasonable.
>>
>
> Sure this is reasonable, and it clearly shows the need to avoid
> systematic calls to the filesystem.
>
> But here's another use case : suppose you have a portal site where 10%
> of the pages make 90% of the requests, but where 100% of the pages are
> monitored, because they have been visited at least once since server
> startup. I'm not good at statistics, but lets suppose this results in
> only 15% of all pages to be requested in a monitor scan period.
>
> This means that at each refresh period, ActiveMonitor will scan 100% of
> the pages when only 15% are really needed to be scanned : 85% of useless
> File.getLastModified() !! This is what I wanted to point out.
My concept of how to use Monitor was to monitor things that require
action. This means that XSP and Sitemap files should be monitored--their
change requires action by the system. Cached resources must be monitored,
as they affect the validity of the cache. The action I foresee for this
event is to invalidate the cache entry and release the Resource.
This way only the pages that are required to be monitored are monitored.
As to the 85% useless File.getLastModified(), you have to consider the
cost of the action. If you have that many resources that you have not
removed your Resource entry, then there is something wrong in the way
you are using it. Also, if this is happening in a background thread
at low priority (read asynchronously that will not override actual
serving of requests), then the cost of these extraneous calls are lowered!
Do you need to monitor resources in a ClassLoader? most likely not. You
merely have to assume that they do not change--which for a production
system is most likely true.
>>>It seems to me that the main benefit of ActiveMonitor is for resources
>>>that are systematically checked at each and every request : IMO, this
>>>should be limited to configuration files and sitemaps.
>>>
>>Yes and also systematically checked are the Generators. On every request
>>the CachingStreamPipeline validates TimeStampCacheValidity. TimeStampCacheValidity
>>is set i.e. from the FileGenerator, which calls getLastModified() every
>>request. TimeStampCacheValidity signals if a Source (i.e sample.xml)
>>has changed or not.
>>
>>
>>>For less-frequently used resources, wouldn't it be a better solution to
>>>only call getLastModified() when the resource is actually used and the
>>>time since the last call to getLastModified() is greater than the
>>>refresh period ? This would be a kind of buffering in front of the
>>>filesystem. Also, can't this be integrated directly in Source ?
>>>
>>Hmm I don't get you here ;).
>>
>
> Ok, maybe some code will be more clear ;)
>
> public class FileResource ... {
> private File;
> private long cachedLastModified = 0;
> private long nextCheckTime = 0;
> private long refreshPeriod = 10000; // configurable
>
> public FileResource(File file) {
> this.file = file;
> refresh();
> }
>
> public long getLastModified() {
> if (System.currentTimeMillis() > nextCheckTime) {
> refresh();
> }
> return cachedLastModified;
> }
>
> private void refresh() {
> nextCheckTime = System.currentTimeMillis() + refreshPeriod;
> cachedLastModified = file.lastModified();
> }
> }
>
> This ensures the LastModified information isn't older that the refresh
> period, and that - and this what I wanted to explain - refresh occurs
> only when the information is actually requested.
>
> If a FileResource is used once a week, File.lastModified() will be
> called only once a week even if refreshPeriod is 1 hour, while in the
> same conditions ActiveMonitor will call it 24*7 = 168 times !
>
> Note also that the above algorithm can really easily be integrated into
> AbstractSource.
>
> Thoughts ?
ActiveMonitor is a tool for developing apps, you might not want it all
the time. Also, ActiveMonitor is best used in systems where action
must be taken on the resource. I would rather the ProgramGenerator
and Cache system uses it, and all other Generators and Transformers,
etc. never make _any_ calls to getLastModified(). That way, if the
cache entry expires, the Resource reference is released, and no other
Generator/Transformer/etc. needs to be notified.
>
>
>>>Last point : your changes in ProgramGenerator make the assumption that
>>>sources are files. This won't be true in unexpanded war files and will
>>>very likely break the engine ;)
>>>
>>But how works the FileGenerator and his related, when they call
>>getLastModified() from the Source in this case?
>>
>
> Using URLConnection.getLastModified(), whose abilities highly depend on
> the protocol handler.
>
>
>>Cheers
>>Gerhard
>>
>
> Sylvain.
>
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, email: [EMAIL PROTECTED]
