thanks to all for the comments and interest. I guess I could make the proposal and proof of concept implementation for geotools. Once uDig can smoothly show a table with thousands of features we could more seriously think on a change request for OGC, but yes, the names and semantics should definately match what's on the csw spec.
Gabriel On Wednesday 02 August 2006 19:54, Javier de la Torre wrote: > Yes! :) > > +10 > > And if uDig supports it then that would be great... I will try to > contact the people from gvSIG and see of they can work on supporting > this too. > > If you want to then submit it to OGC and I can provide you with some > scenarios where this is a MUST. Specially if what you want to do is > analysis with the data. > > Thanks. > > Javier. > > On 01/08/2006, at 22:40, Chris Holmes wrote: > > I know Javier would love to have this feature exposed in WFS. > > > > I'm more or less +1. I'd say check out the C-SW 2 spec if you > > haven't already as I believe they do paging, and you should use the > > same names as they do. We could probably expose through WFS as an > > extension of some sort, and submit a change request as well. > > > > Chris > > > > Gabriel Roldán wrote: > >> Jessy and I were ranting about adding the ability to the GeoTools > >> DataStore api of being required for a page of data/features, > >> rather than the whole dataset. > >> That would immensely improve (and actually allow) some common > >> scenarios like presening a result set in tabular form. > >> On the client side (uDig in this case, but could be JSF, swing, > >> whatever), it is easy to, for example, implement a lazy List, as > >> long as the underlying data api allows for pagination. > >> Doing so in geotools would be actually easy! > >> No API change would be needed beyond adding two fields to Query: > >> fromIndex and pageSize. OrderBy is already present in the Filter > >> spec. > >> here is the conversation, hope to get some comments. > >> Gabriel > >> ----------------------- > >> [18:40:00]Gabriel Roldán was thinking what should be needed on the > >> geotools front in order to make TableView lazily loaded. On the > >> uDig front it is easy, problem would be with the insufficient > >> geotools Data api > >> [18:40:50] … I thought the DataStore api could be augmented in a > >> similar fashion as the catalog queries are done, aka, you can > >> request by "pages" of data > >> [18:40:56] Jesse Eichar I think Jody envisioned using the > >> FeatureList API > >> [18:41:01] … you can get all the fids. > >> [18:41:16] … then using the FeatureList API load up the features > >> by fid > >> [18:42:27] Gabriel Roldán I mean, on the uDig side you can use > >> FeatureList for sure. The lazy loading strategy would be > >> transparent for uDig, but still the datastore api should be more > >> friendly > >> [18:42:35] … getting all the fids could still be a pain > >> [18:43:02] Jesse Eichar I see. > >> [18:43:12] Gabriel Roldán an approach that's proven to work is > >> getting the feature/element count and then request by pages of a > >> user defined size > >> [18:43:42] … (proven to work == I'm doing it on the catalog > >> implementation) > >> [18:43:54] Jesse Eichar I can't think of any problems with that > >> right off. > >> [18:44:25] … Where would that fit in? > >> [18:44:34] … WE have a feature collection > >> [18:44:38] … feature list.. > >> [18:44:54] … would you add a method to feature store? [18:45:06] … > >> New type of query? > >> [18:45:10] Gabriel Roldán you should treat FeatureList as a normal > >> list, use the get(int) or iterator() as normal > >> [18:45:38] … the featurelist impl uses a paging strategy to > >> retrieve content on the back > >> [18:46:19] … provided that you pass it the Query and the list size > >> in the constructor, for example > >> [18:46:58] Jesse Eichar Makes sense. Would you add a new method > >> in FeatureSource? getFeatures(Query, pageSize) > >> [18:46:57] … ? > >> [18:47:18] … or make a new type of Query that has that information. > >> [18:47:29] Gabriel Roldán getFeature(Query, startIndex, pageSize) > >> [18:48:26] Jesse Eichar I think I prefer that too. But could be > >> hard to get it to fly because it will eventually require > >> negotiation with GeoAPI. > >> [18:49:11] Gabriel Roldán Well, could encapsulate it in Query as > >> well, after all Query _is_ a parameter object > >> [18:50:14] … and I guess something like that could certainly be in > >> future versions of wfs spce at least, since they already > >> recognized the problem and designed it for catalog 2.0 > >> [18:50:57] Jesse Eichar I didn't know that. > >> [18:53:52] Gabriel Roldán now, implementing getFeatures(Query, > >> from, size) or whatever would have its implications. Some RDBMS > >> backed datastores could manage it easyly I guess > >> [18:54:32] Jesse Eichar It does. [18:54:34] … WFS for example > >> [18:55:16] … The problem is that things aren't inherently ordered > >> [18:55:46] … so index 3 could (at least theoretically) be a > >> different item between calls. > >> [18:56:31] … WFS 1.1 I think has some sort-by functionality I > >> think but 1.0 doesn't > >> [18:57:49] … For that one it makes sense to get all fids in the > >> query. Shapefile and other file based ones I the will be ok. > >> [18:58:13] Gabriel Roldán just 1' > >> [18:58:39] Jesse Eichar sur > >> [19:00:34] Gabriel Roldán sorry, had a phone call > >> [19:00:42] Jesse Eichar np > >> [19:00:44] Gabriel Roldán you're completelly true > >> [19:01:10] … so a requirement would be an order being explicitly > >> set in the Query > >> [19:01:37] Jesse Eichar For WFS we could obtain all fids and > >> manage the paging on the client, at least until 1.1. [19:01:39] > >> Gabriel Roldán what I'm doing in catalog is ordering by ID if the > >> Query has no orderBy > >> [19:02:05] Jesse Eichar we can order by fid if not specified. > >> [19:02:14] … seems reasonable. > >> [19:02:19] Gabriel Roldán problem with fids is that getting two > >> million fids could still be quite killer > >> [19:03:07] Jesse Eichar I know it. I'm open to suggestions... > >> [19:03:09] Gabriel Roldán what raises me another concern I was > >> thinking on > >> [19:03:41] … I know we've defined feature ids to be String as to > >> be friendly with the WFS spec > >> [19:03:58] … still it makes no much sense on the pure java side of > >> things > >> [19:04:10] … I would like to see FID as an interface > >> [19:04:33] … so implementors could optimize as needed, instead of > >> creating millions of strings by prepending the feature type name, etc > >> [19:05:07] … but that's another concern, I tend to ramp :P > >> [19:05:16] Jesse Eichar :D You best jump on the FM discussion with > >> that. I don't think that's going to happen too soon. > >> [19:05:34] Gabriel Roldán yeah, I guess so > >> [19:06:11] Jesse Eichar But back to the point. I'm completely in > >> agreement with you on the Paging requirement. > >> [19:06:30] … I'd be happy to do some of the implementations for you. > >> [19:06:49] Gabriel Roldán cool, that kind of stratagies works just > >> great for presenting huge amounts of tabular data in other domains > >> [19:06:56] … so it should work for us too > >> [19:07:19] Jesse Eichar I think it has to be done. Its impossible > >> to deal with this amount of data otherwise. > >> [19:07:47] Gabriel Roldán I like the idea that the FeatureSource > >> interfaces doesn't needs to be touched > >> [19:07:53] … just Query > >> [19:08:19] Jesse Eichar It'll be much quicker and easier to get it > >> integrated with geotools that way. Pretty clean too. [19:08:19] > >> Gabriel Roldán we already have order by in Filter, so just from > >> and page size are needed > >> [19:09:12] … note that FeatureCollection.size() still should > >> return the whole query size (aka, hits), and not the page size > >> [19:09:31] Jesse Eichar Yes. > >> [19:09:47] Gabriel Roldán in that case I'm wondering what's the > >> easiest way of knowing when you're done fetching content > >> [19:09:51] Jesse Eichar and get() can get any feature, not just > >> those in the current page (for FeatureList) > >> [19:09:57] Gabriel Roldán other by requiring client code to use a > >> counter > >> [19:10:02] … sure > >> [19:10:32] … I did that for presenting catalog results using Java > >> Server Faces and works great > >> [19:10:57] … a custom list impl that queries the required page of > >> data if the index isn't on the current page > >> [19:11:38] Jesse Eichar Do you cache the fetched features so you > >> don't have to get them more than once? or get a fresh copy each > >> time? > >> [19:12:36] Gabriel Roldán in the catalog case I fetch the whole > >> page onto memory. In our case I guess we could be even smarter and > >> maintain the streamed nature of stuff even on the pages > >> [19:13:12] … not sure if I'm explaining me well enough > >> [19:13:25] Jesse Eichar You're doing fine > >> [19:16:02] Gabriel Roldán cool, do you mind if I post this to the > >> list? > >> [19:16:15] Jesse Eichar No at all > >> [19:16:26] Gabriel Roldán better said, do you think it is > >> something that worths being posted? > >> [19:16:37] Jesse Eichar haha > >> [19:16:44] … Yeah I think it should be. > >> [19:16:56] … People will have comments for sure. > >> [19:17:34] Gabriel Roldán nice, forgot to jump on the geoserver > >> irc meeting, uhg > >> [19:17:49] Jesse Eichar shoot me too > > > > -- > > Chris Holmes > > The Open Planning Project > > http://topp.openplans.org > > <cholmes.vcf> > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your opinions on IT & business topics through brief surveys -- and earn > cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Geotools-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/geotools-devel -- Gabriel Roldán ([EMAIL PROTECTED]) Axios Engineering (http://www.axios.es) Tel. +34 944 41 63 84 Fax. +34 944 41 64 90 ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Geotools-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geotools-devel
