On Mon, Feb 24, 2020 at 5:46 AM Sexton Michael <[email protected]> wrote:
> Hi Geoserver Users, > > We’ve tested many things, such as ingesting the entire WFS and running > operations to pull out and cache the values we want on a periodic basis. > This can put tremendous strain on our instances, resulting in denial of > service attacks being experienced. We have also tried directly connecting > to the database, but this only works for service endpoints we directly > control. > > > > We have test existing functionality in GeoServer, but they don’t address > the issues experienced, they include: > > > > 1. GeoServer provides an operation to retrieve all properties for > an attribute, which only works for WFS 2.0, GetPropertyValue. This provides > a full list of non-unique values, which means further data operations are > required, and typically does not perform well for large dataset. > As far as I know, the WFS protocol has no way to extract a list of unique values. > 2. There is also a WPS function in GeoServer, gs:Unique, which > returns the unique values for the given attribute, but seems to ingest the > WFS first, then crunch the unique values. Again, it typically does not > perform well for a large dataset. > The code tries to give the opportunity to the underlying data store to optimize the calculation, if the backend is a database the request should be turned into a "select distinct(...)". If that does not work for you, there might be a delegation bug, the stores have a set of wrappers around them, like renaming, reprojecting, security and so on, that might fail to delegate down the unique visitor. If the store is not a relational database (like WFS cascading)... then yes, indeed there is no optimization, all the data will be loaded and then unique values will be calculated. Each store has an opportunity to optimize, but code needs to be written, and the actual backend needs to provide a fast way to compute the unique values. Cheers Andrea == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- *Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.*
_______________________________________________ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-users
