Hi Geoserver Users, We're developing web services which provide access to large volumes of geoscientific data, in excess of 1 million rows, often with up to 300 attributes. To be able to dynamically query these web services, our UI developers have built filters to provide lists of values that reflect the underlying dataset, often dynamically after filters have already been applied.
An example user story is as such Our web UI developers: * want to query a web service recover distinct values for any property of any layer of any WFS web service * want to query the web service using an OGC Filter either in a WFS GetFeature request or SLD sent to a WMS GetMap. * want to input parameters to the distinct layers, each of which may be unique to the layer, and responsive to other requests we have made to the web service, without having to hardcode the parameters in our UI * want to acquire the parameters in a consistent and performant way We've tested many things, such as ingesting the entire WFS and running operations to pull out and cache the values we want on a periodic basis. This can put tremendous strain on our instances, resulting in denial of service attacks being experienced. We have also tried directly connecting to the database, but this only works for service endpoints we directly control. We have test existing functionality in GeoServer, but they don't address the issues experienced, they include: 1. GeoServer provides an operation to retrieve all properties for an attribute, which only works for WFS 2.0, GetPropertyValue. This provides a full list of non-unique values, which means further data operations are required, and typically does not perform well for large dataset. 2. There is also a WPS function in GeoServer, gs:Unique, which returns the unique values for the given attribute, but seems to ingest the WFS first, then crunch the unique values. Again, it typically does not perform well for a large dataset. Is there anything else we should try that I may not be aware of? Kind Regards Michael Sexton Geoscience Australia Disclaimer: This e-mail (and files transmitted with it) is intended only for the person or entity to which it is addressed. If you are not the intended recipient, then you have received this e-mail by mistake and any use, dissemination, forwarding, printing or copying of this e-mail and its file attachments is prohibited. The security of emails transmitted cannot be guaranteed; by forwarding or replying to this email, you acknowledge and accept these risks. -------------------------------------------------------------------------------------------------------------------------
_______________________________________________ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-users
