Justin Deoliveira ha scritto: > >> >> The FileDataStoreSpi is badly designed for a number of reasons: >> - it assumes you deal with certain extensions. This is overrated, the >> datastore could operate by inspecting magic numbers in the file header >> instead of using extensions >> - it assumes you can create a datastore with a url alone. Nope, we want >> a namespace too >> - it assumes there is only one feature type per url. Nope, a file can >> be complex and contain many layers >> >> Since I wanted to work on this both in trunk and 2.5.x I tried hard >> to stay away from any mandatory new interface and work with what's >> already there. Any datastore dealing with files has at least to >> be able and handle a url and a namespace, so that's just as much >> I'd like to assume. >> >> What would be useful, but I don't consider mandatory, is a FileDataStore >> interface that provides a list of files captured by a certain datastore, >> so that I can avoid scanning them. I can workaround the lack of it >> by forcing the registration of a single feature type just once, with >> the first datastore that can handle it. When I open a file, I'll >> grab all the feature types inside of it, in the case of property data >> store for example, that will mean all the property files in the >> current directory (that's how property data store works if my memory >> serves me right). > > Ok, fair enough, but still confused about how you can detect if a > datastore works with files. Is the approach to assume that if the > datastore can handle files it has a "url" parameter with a type of > java.io.File?
Nope. GeoServer treats any datastore accepting a parameter of type java.net.URL as a file based one already. Now, in the case of directory data store, we know that the URL is by construction a local file, so if anything can open it, we're good. Hmmm... actually you have a point, PropertyDataStore takes a File instead... there is probably a converter at work I guess, otherwise I don't know how the GeoServer UI can work. Oh, I could go for an extra interface, the problem is that to avoid making shapefile datastore and friends depend onto an unsupported method, I would have to locate that interface in main, that would be new API in a stable branch... Coding this only on trunk to wait 6 months before users can hit it is... wrong. In 6 months I'll probably have forgot half of how the datastore works. And I cannot escape by making this a GeoServer only extension since I would still need to have the datastores implement the new interface... >> >>>> Issues: >>>> - a certain datastore can open multiple files (shapefile, property ds), >>>> we want to avoid keeping duplicate datastores around >>>> - a directory (or worse, a tree) can hold a massive amount of feature >>>> types, there are legitimate scalability/memory consumption concerns. >>>> >>>> Using a lightweight (soft reference based) cache has issues with >>>> datastore disposal, as the datastore we're trying to dispose might >>>> be the holder or a resource that might be in use by a reader or a >>>> feature >>>> source, closing it might kill the current user... >>>> This one is hard actually, the api does not give us any clue on >>>> whether a datastore generated object is still being used or not... >>>> To avoid it we'd have to keep strong references to all datastores >>>> that have returned a feature source, a reader or a writer at >>>> least once. Maybe we can add a custom api to this datastore to >>>> force some resource release (stop gap measure for the lack of a better >>>> way). >>>> >>> Yeah tricky. It seems to me what we lack is a dispose on FeatureSource. >> >> Hummm.. yeah, but I fear this will escalate badly codebase refactoring >> wise. >> If we add that, we'll also have to start keeping references to all >> resource holders provided to the clients >> inside the DataStore itself. This accounts not only for FeatureSource, >> but also for collections, subcollections, readers, iterators, >> writers... given the variety of ways a datastore has been created in >> the past >> years, the mind boggles at the thought. >> I'd say we first simplify a little bit the datastore interfaces (die >> feature collection, die!) and then we can think hanging ourselves >> with nice resource tracking? > Yup, its messy. But perhaps a first cut we could assume that client is > well behaved and does not dispose a FeatureSource while iterators are > still open on collections? That way we just have to worry about calling > dispose() on FeatureSource when we are done with it. And if we have been > bad we will find out about it. It would theoretically work in GeoServer > would it not? > > Anyways, i am fine to push it back and just for a first pass use a hard > cache... or perhaps just limit the number of sub datastores or something. Adding FeatureSource.dispose() is an API change, it requires a GSIP, can be done only on trunk. It turns the work of two days in the one of two weeks and sticks the work only on trunk (along with the above interface). It moves the priority of this work in the long term scheduling, not as a feature for GS 1.7.2, so I will have to renegotiate both the new scope (bigger) and the new scheduling. Which is kind of fun, because it will make it be delayed just like the other two times I attempted to do anything about this (for GeoServer 1.4.x and GeoServer 1.5.x, for 1.6.x I simply considered it un-salvageable). Can we hit a compromise of any kind so that something can be done on 2.5.x? My enthusiasm on this directory datastore remake has suddenly dropped below zero, as I've seen this movie already... Cheers Andrea -- Andrea Aime OpenGeo - http://opengeo.org Expert service straight from the developers. ------------------------------------------------------------------------------ SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada. The future of the web can't happen without you. Join us at MIX09 to help pave the way to the Next Web now. Learn more and register at http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/ _______________________________________________ Geotools-devel mailing list Geotools-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
