Tamas,
I think some of the generic issues involved with a cache are caste into
the context of this RFC follow. I think answers to questions like this
will help the rfc provide a high level overview of how this will work
and deal with specific cache related issues.
1. How large can it get? Can I configure a max size? How?
2. What happens to the behavior when it fills up?
3. How are items expired from the cache?
4. What happens when the cache gets fragmented?
5. What happens when there is not room in the cache to add to it? Either
it is fragmented or full?
6. There has been some implementation of layer/feature? security, I
thought, how will the cache deal with that?
7. Can/do items have a Expiration time set on them? after which they are
invalid and can be purged or assumed to be purged?
8. Is a cache - mapfile specific or mapserver instance specific or user
session specific? How will this be handled from a config and
installation point of view? In a live production site with multiple
users, multiple mapfiles, and multiple mapserver instances how does all
this work? How to the requests get to their respective cache instance.
Answers to many of these questions probably represent config options for
the cache, where would these be configured on the installation server?
I did not check the rfc, you may have addressed some of these already,
this is a dump of the problems I have had to consider working with other
caching applications.
-Steve W
Tamas Szekeres wrote:
Frank,
I'm not really completely certain how it would work in practice though.
I'm not sure it would be good to keep around features fetched for drawing
purposes. I had in the past assumed we would just store features instead
of feature ids in the resultCacheObj - in which case the caching would
only affect queries, and wouldn't increase memory use during the normal
render loop.
I would support caching not only for the feature query but also for
the subsequent
renderings as well. Therefore at least the features of the previous
rendering would be
retained in the cache. Alternatively the user could configure the
adapter for fetching
features for an enlarged rendering area or for the entire area covered
by the layer.
The RFC for the feature cache will address the applications being
capable to maintain
long lived internal reference to the map object. This kind of
application will tolerate the
increased memory footprint in exchange for the higher rendering and query
performance.
I would also support adapter dependent custom actions to be performed
using the
mapscript API such as loading the features into the cache. This action
could be
implemented thread safely so prefetching the features could be done even
from a
background thread.
Note, I was suggesting that "adaptor layers" would reference another
raw feature data source layer by name, rather than nesting it within
the layerObj. This is analygous to how the tileindexes are referenced
from other layers.
At this point I cannot figure out how this approach would be realized.
From the user's
perspective how it will affect the mapfile configuration and the
mapscript API. Will it
change the API in a logical or rather a constrainted manner?
And how much amount of code should be changed to implement this feature?
> Theoretically I would write a builtin support for caching features
> from any of the
> providers but i would not want to implement a new provider eg.
> MS_FEATURECACHE_PROVIDER along with MS_OGR or the others. This problem
> requires to extend the functionality of an existing provider not to
> write a new one.
I'm afraid I didn't understand this.
Well I have just mentioned that I could not imagne the suggested
solution easily.
But now, I would consider a solution providing to specify one or more
layer being contained
by another layer. This nested approach could be implemented for the
mapfile configuration
and the mapscript API as well like:
LAYER
CONNECTIONTYPE CUSTOM_LAYER
...
LAYER
CONNETIONTYPE OGR
...
END
LAYER
CONNECTIONTYPE POSTGIS
END
END
I'm considering this solution is different from my proposal, and would
require to change a similar
amount of the existing code. This approach would give a solution for
different problems
(like implementing provider independent joins for example). Recall
that originally I would like
to add functionality to and existing provider. I can see a slight
analogue to the difference between
the inheritance and the containment from the OO terminology.
I'm not sure why binding the work into the core mapserver code is
more effective. When I spoke of Mapscript, I was meaning a case
where mapscript is used explicitly for drawing the features. So
a mapscript loop can query the features from the layer, and then render
each with appropriate per-feature draw calls. In this "mapscript owns
the rendering loop" approach any arbitrary transformation can be
applied.
Do you mean creating an inline layer and feeding the features into it
on the fly?
How can the implementor provide the reusability and the configurability of
the implementation if needed?
Best Regards,
Tamas
