Hi, Currently, GeoServer has the flag skipNumberMatched for FeatureTypeInfo, which can be used to disable calculating the number of matched results. So what I'm proposing is not at all new for GeoServer. However there are still some places (the ones I mentioned) where GetFeature might calculate the number of features even when this flag is enabled and the information is not necessary to enforce paging semantics. By my logic think we can safely skip counting features in those two places when skipNumberMatched is true and there is only one query in the GetFeature request (=because we don't need to figure out which queries and offsets should be included in this page).
If you can confirm my hypothesis, this issue / patch can be treated as just an improvement to the existing skipNumberMatched feature that does not change query / paging semantics in any shape or form. Sampo On Sun, Nov 15, 2015 at 11:20 AM, Andrea Aime <[email protected]> wrote: > Hi Sampo, > if I understand your proposal correctly, you're suggesting to break both > standard and backwards compatibility, which > is not something I'm too keen to accept :-p > Unless I'm misunderstanding... been reviewing/fixing/merging pull requests > all weekend and I'm pretty tired by now > (anyone reading: really need some help there btw). > > The "number matched" count is available on in WFS 2.0, it's a second count > that's used to notify the total number > of matched features in a paging situation. > > So... we cannot have it skip also normal feature counting, that has a > different meaning. > > I checked if there is a way not to report the feature count, stating the > value > is "unknown". As far as I can see, in WFS 1.1 the numberOfFeatures > attribute is optional: > http://schemas.opengis.net/wfs/1.1.0/wfs.xsd > > However, it seems that in wfs 2.0 there is no salvation, the local count > is mandatory: > http://schemas.opengis.net/wfs/2.0/wfs.xsd > In particular: > <xsd:attribute name="numberMatched" type="wfs:nonNegativeIntegerOrUnknown" > use="required"/> > <xsd:attribute name="numberReturned" type="*xsd:nonNegativeInteger*" > *use="required"*/> > > Another possible approach for very slow simple feature collections should > be to dump > them on disk to avoid hitting the database again, it's something we do in > sorting/z-ordering in geotools for shapefiles > and friends, using a highly optimized binary feature serializer: > > https://github.com/geotools/geotools/blob/master/modules/library/main/src/main/java/org/geotools/data/sort/SimpleFeatureIO.java > > The sorting logic keeps a certain amount of features in memory (up to 1000 > normally), past that they get dumped on disk > instead. Could be an option I guess, but yep, it would still require an > extra flag in FeatureTypeInfo. > > Cheers > Andrea > > > > On Thu, Nov 12, 2015 at 3:31 PM, Sampo Savolainen < > [email protected]> wrote: > >> Hi, >> >> I recently created a ticket with a pull request to make GetFeature >> queries faster for data stores where counting features is slow. >> https://osgeo-org.atlassian.net/browse/GEOS-7301 >> >> In my pull request I proposed adding a new field to FeatureTypeInfo that >> would allow GeoServer to skip counting features altogether (under certain >> conditions) that would make things faster, but break strict compatibility >> with some types of queries. However I only later noted there is a flag >> already which does this in part: skipNumberMatched. I updated the ticket >> with a new proposal that I'ld like to get some comments on: >> >> >> "I was looking at the existing flag in FeatureTypeInfo, >> skipNumberMatched. The implementation looks very close to what I'm trying >> to achieve here, but the current accepted implementation does not make >> GeoServer skip counting all features. More specifically, I'm referring to >> these rows of code: >> >> >> https://github.com/geoserver/geoserver/blob/2e19a4ebe7f9c3698ac3eecf002da76bff0aa506/src/wfs/src/main/java/org/geoserver/wfs/GetFeature.java#L531 >> >> >> https://github.com/geoserver/geoserver/blob/2e19a4ebe7f9c3698ac3eecf002da76bff0aa506/src/wfs/src/main/java/org/geoserver/wfs/GetFeature.java#L552 >> >> Would you be willing to accept a patch where those counts are skipped if >> both: >> >> - primary feature of the query has skipNumberMatched = true, and >> - there is only a single query in the GetFeature request (paging >> >> .. ? >> >> This would simplify this patch to only affecting GetFeature.java and >> remove the need for a new field in FeatureTypeInfo." >> >> Can you think of any new problems with this approach (meaning that I >> should stick with adding a new flag to FeatureTypeInfo) or should I update >> my pull request based on this idea? >> >> >> Sampo >> >> -- >> Sampo Savolainen >> Managing Director, Spatineo Oy >> [email protected] >> +358-407555649 >> Linnankoskenkatu 16 A 17, 00250 Helsinki, Finland >> www.spatineo.com, twitter.com/#!/spatineo >> www.linkedin.com/company/spatineo-inc >> >> This message may contain privileged and/or confidential information. If >> you >> have received this e-mail in error or are not the intended recipient, you >> may not use, copy, disseminate, or distribute it; do not open any >> attachments, delete it immediately from your system and notify the sender >> promptly by e-mail that you have done so. >> >> >> ------------------------------------------------------------------------------ >> >> _______________________________________________ >> Geoserver-devel mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >> >> > > > -- > == > GeoServer Professional Services from the experts! Visit > http://goo.gl/it488V for more information. > == > > Ing. Andrea Aime > @geowolf > Technical Lead > > GeoSolutions S.A.S. > Via Poggio alle Viti 1187 > 55054 Massarosa (LU) > Italy > phone: +39 0584 962313 > fax: +39 0584 1660272 > mob: +39 339 8844549 > > http://www.geo-solutions.it > http://twitter.com/geosolutions_it > > *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003* > > Le informazioni contenute in questo messaggio di posta elettronica e/o > nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il > loro utilizzo è consentito esclusivamente al destinatario del messaggio, > per le finalità indicate nel messaggio stesso. Qualora riceviate questo > messaggio senza esserne il destinatario, Vi preghiamo cortesemente di > darcene notizia via e-mail e di procedere alla distruzione del messaggio > stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, > divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od > utilizzarlo per finalità diverse, costituisce comportamento contrario ai > principi dettati dal D.Lgs. 196/2003. > > > > The information in this message and/or attachments, is intended solely for > the attention and use of the named addressee(s) and may be confidential or > proprietary in nature or covered by the provisions of privacy act > (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection > Code).Any use not in accord with its purpose, any disclosure, reproduction, > copying, distribution, or either dissemination, either whole or partial, is > strictly forbidden except previous formal approval of the named > addressee(s). If you are not the intended recipient, please contact > immediately the sender by telephone, fax or e-mail and delete the > information in this message that has been received in error. The sender > does not give any warranty or accept liability as the content, accuracy or > completeness of sent messages and accepts no responsibility for changes > made after they were sent or for other risks which arise as a result of > e-mail transmission, viruses, etc. > > ------------------------------------------------------- > -- Sampo Savolainen Managing Director, Spatineo Oy [email protected] +358-407555649 Linnankoskenkatu 16 A 17, 00250 Helsinki, Finland www.spatineo.com, twitter.com/#!/spatineo www.linkedin.com/company/spatineo-inc This message may contain privileged and/or confidential information. If you have received this e-mail in error or are not the intended recipient, you may not use, copy, disseminate, or distribute it; do not open any attachments, delete it immediately from your system and notify the sender promptly by e-mail that you have done so.
------------------------------------------------------------------------------
_______________________________________________ Geoserver-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-devel
