Re: [Geoserver-users] Features-templating and OGC services, filters not working, errors

[Henning Lorenz]

Hi Ian,

The problem is likely related to the features templating plug in, no 
problems with the demo layers. All of my other layers on the same server 
use only app-schema and have no problem with the combination of bbox and 
json.


Maybe somebody else who has a working layer with features-templating 
could verify by testing whether they experience the same problem:


Only BBOX works:

.../ows?service=WFS=2.0.0=GetFeature==18,60,20,65

Only outputFormat json works:

.../ows?service=WFS=2.0.0=GetFeature==application/json

Bpth in one query - error:

.../ows?service=WFS=2.0.0=GetFeature==18,60,20,65=application/json

Henning


On 2024-01-26 16:10, Ian Turton wrote:



On Fri, 26 Jan 2024 at 14:51, Henning Lorenz 
 wrote:


Thanks, Julian!

Your comments helped a lot. With this, the problem is reduced to
the  not working in WFS 2.0.0. I went back to OGC
documentation and started out with a query from the documentation
- and it works. It turns out that the problem is  in
combination with =application/json. Either on its own
works fine and as expected. Both combined in the query result in a
java runtime exception "Unable to evaluate template path against
the template".


That does sound like a bug, can you open a ticket with an example 
query against one of the demo layers please


Thanks

Ian

Henning

On 2024-01-26 14:00, Julian Hollingbery wrote:


Hi,

You are mixing parameters between different versions of OGC
protocols, and this is causing at least some of your issues.

  *  is a WFS version 1.x parameter. It should not do
anything in a WMS request. For WFS 2.0.0, the corresponding
parameter is called 
  *  is technically only a WFS-parameter, so for it to
work in WMS is a non-standard feature in GeoServer.

/Julian

    *From:*Henning Lorenz 
<mailto:henning.lor...@geo.uu.se>
*Sent:* Thursday, 25 January 2024 11.16
*To:* geoserver-users@lists.sourceforge.net
*Subject:* [Geoserver-users] Features-templating and OGC
services, filters not working, errors

Geoserver 2.24 and 2.21 snapshot with module app-schema and
community modules features templating, ogcapi-features (all from
same build date)

Hello,

I have set up a seemingly functional layer with features
templating. However, I encounter a rather odd behaviour when I
try to include filters in the query URLs. Here is a summary (I
paste examples below):

WMS 1.3 requests: Simple GetMap request works. 
works. CQL-filters do not work. OGC-filters on attributes work.
Spatial OGC-filter returns an empty map.

WFS 2.0.0 query: Simple Get Feature query works.  has
no effect.  returns an error. CQL filters do not work.
OGC-filters on attributes work. Spatial OGC-filter returns error.

No spatial filtering at all, no CQL filters, only OGC filters on
attributes work. This limits the usefulness of WMS, and WFS is
almost completely useless. I guess that something is wrong, but
do not even know where to start looking for the problem, assuming
that there is a common underlying problem. The log entries are
very peculiar - not much with CQL-filter errors. On the other
hand, extensive warnings that the OGC-filter - which actually
works - could not be parsed. java.lang.RuntimeException on the
spatial filter errors (java.lang.NullPointerException on OGC
filter; Unable to evaluate template path against the template on
bbox).

The main issue is that spatial filtering is not working (either
no effect or returns error) while all features are displayed in
the correct locations in a basic WMS map. Very peculiar is also
that basic limitations like maxFeatures do not work, as it has
nothing to do with the actual content. Any ideas on how to
proceed with troubleshooting are very much appreciated.

Thank you and best wishes,

Henning

---

Below the tested queries with short comments:

WMS 1.3:

Requesting a map without any additional parameters works as expected:

http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=image/jpeg

<http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=image/jpeg>

maxFeatures works:

http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=5=62,21,65,26=700=600=EPSG:4326=image/jpeg

<http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=5=62,21,65,26=700=600=EPSG:4326=image/jpeg>

CQL-filter on an attribute does not work (displays map with all
features, unfiltered, as without filter):

http://localhost:8080/geoserver/eposgo/

Re: [Geoserver-users] Features-templating and OGC services, filters not working, errors

[Henning Lorenz]

Thanks, Julian!

Your comments helped a lot. With this, the problem is reduced to the 
 not working in WFS 2.0.0. I went back to OGC documentation and 
started out with a query from the documentation - and it works. It turns 
out that the problem is  in combination with 
=application/json. Either on its own works fine and as 
expected. Both combined in the query result in a java runtime exception 
"Unable to evaluate template path against the template".


Henning

On 2024-01-26 14:00, Julian Hollingbery wrote:

Page Title

Hi,

You are mixing parameters between different versions of OGC protocols, 
and this is causing at least some of your issues.


  *  is a WFS version 1.x parameter. It should not do
anything in a WMS request. For WFS 2.0.0, the corresponding
parameter is called 
  *  is technically only a WFS-parameter, so for it to work in
WMS is a non-standard feature in GeoServer.

/Julian

*From:*Henning Lorenz 
*Sent:* Thursday, 25 January 2024 11.16
*To:* geoserver-users@lists.sourceforge.net
*Subject:* [Geoserver-users] Features-templating and OGC services, 
filters not working, errors


Geoserver 2.24 and 2.21 snapshot with module app-schema and community 
modules features templating, ogcapi-features (all from same build date)


Hello,

I have set up a seemingly functional layer with features templating. 
However, I encounter a rather odd behaviour when I try to include 
filters in the query URLs. Here is a summary (I paste examples below):


WMS 1.3 requests: Simple GetMap request works.  works. 
CQL-filters do not work. OGC-filters on attributes work. Spatial 
OGC-filter returns an empty map.


WFS 2.0.0 query: Simple Get Feature query works.  has no 
effect.  returns an error. CQL filters do not work. OGC-filters 
on attributes work. Spatial OGC-filter returns error.


No spatial filtering at all, no CQL filters, only OGC filters on 
attributes work. This limits the usefulness of WMS, and WFS is almost 
completely useless. I guess that something is wrong, but do not even 
know where to start looking for the problem, assuming that there is a 
common underlying problem. The log entries are very peculiar - not 
much with CQL-filter errors. On the other hand, extensive warnings 
that the OGC-filter - which actually works - could not be parsed. 
java.lang.RuntimeException on the spatial filter errors 
(java.lang.NullPointerException on OGC filter; Unable to evaluate 
template path against the template on bbox).


The main issue is that spatial filtering is not working (either no 
effect or returns error) while all features are displayed in the 
correct locations in a basic WMS map. Very peculiar is also that basic 
limitations like maxFeatures do not work, as it has nothing to do with 
the actual content. Any ideas on how to proceed with troubleshooting 
are very much appreciated.


Thank you and best wishes,

Henning

---

Below the tested queries with short comments:

WMS 1.3:

Requesting a map without any additional parameters works as expected:
http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=image/jpeg 
<http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=image/jpeg>


maxFeatures works:
http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=5=62,21,65,26=700=600=EPSG:4326=image/jpeg 
<http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=5=62,21,65,26=700=600=EPSG:4326=image/jpeg>


CQL-filter on an attribute does not work (displays map with all 
features, unfiltered, as without filter):
http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=surveyname='1998'=image/jpeg 
<http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=surveyname='1998'=image/jpeg>


OGC-filter on an attribute works as expected:
http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326= 
<http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=>xmlns="http://www.opengis.net/ogc; <http://www.opengis.net/ogc> 
xmlns:gml="http://www.opengis.net/gml; 
<http://www.opengis.net/gml>>surveyname1998=image/jpeg


OGC spatial filter returns empty map (tested with both latlon and 
lonlat, as the order of coordinates in different formats/standards 
(and their combination) always appears to be a mess to me)
http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326= 
<http:

Re: [Geoserver-users] Features-templating and OGC services, filters not working, errors

[Henning Lorenz]

Thanks, Brad!

You are right - what a stupid mistake, which I then copied on and on without 
detecting the simple problem. I can confirm that CQL-Filters work as expected, 
including spatial filters (after I fixed another issue in the query).

Although most tasks can be achieved with CQL-filters, the issues with , 
, spatial OGC filters remain, at least for those users who do not know 
about it. Thus, I am still looking for a solution.

Henning

On 2024-01-25 11:43, Brad Hards wrote:
I didn't test, but maybe CQL_FILTER rather than CQL-FILTER ?









När du har kontakt med oss på Uppsala universitet med e-post så innebär det att 
vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du 
läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/

E-mailing Uppsala University means that we will process your personal data. For 
more information on how this is performed, please read here: 
http://www.uu.se/en/about-uu/data-protection-policy
___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

[Geoserver-users] Features-templating and OGC services, filters not working, errors

[Henning Lorenz]

Geoserver 2.24 and 2.21 snapshot with module app-schema and community modules 
features templating, ogcapi-features (all from same build date)

Hello,

I have set up a seemingly functional layer with features templating. However, I 
encounter a rather odd behaviour when I try to include filters in the query 
URLs. Here is a summary (I paste examples below):

WMS 1.3 requests: Simple GetMap request works.  works. CQL-filters 
do not work. OGC-filters on attributes work. Spatial OGC-filter returns an empty 
map.

WFS 2.0.0 query: Simple Get Feature query works.  has no effect. 
 returns an error. CQL filters do not work. OGC-filters on attributes work. 
Spatial OGC-filter returns error.

No spatial filtering at all, no CQL filters, only OGC filters on attributes 
work. This limits the usefulness of WMS, and WFS is almost completely useless. 
I guess that something is wrong, but do not even know where to start looking 
for the problem, assuming that there is a common underlying problem. The log 
entries are very peculiar - not much with CQL-filter errors. On the other hand, 
extensive warnings that the OGC-filter - which actually works - could not be 
parsed. java.lang.RuntimeException on the spatial filter errors 
(java.lang.NullPointerException on OGC filter; Unable to evaluate template path 
against the template on bbox).

The main issue is that spatial filtering is not working (either no effect or 
returns error) while all features are displayed in the correct locations in a 
basic WMS map. Very peculiar is also that basic limitations like maxFeatures do 
not work, as it has nothing to do with the actual content. Any ideas on how to 
proceed with troubleshooting are very much appreciated.

Thank you and best wishes,

Henning

---

Below the tested queries with short comments:

WMS 1.3:

Requesting a map without any additional parameters works as expected:
http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=image/jpeg

maxFeatures works:
http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=5=62,21,65,26=700=600=EPSG:4326=image/jpeg

CQL-filter on an attribute does not work (displays map with all features, 
unfiltered, as without filter):
http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=surveyname='1998'=image/jpeg

OGC-filter on an attribute works as expected:
http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=http://www.opengis.net/ogc; 
xmlns:gml="http://www.opengis.net/gml;>surveyname1998=image/jpeg

OGC spatial filter returns empty map (tested with both latlon and lonlat, as 
the order of coordinates in different formats/standards (and their combination) 
always appears to be a mess to me)
http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=http://www.opengis.net/ogc; 
xmlns:gml="http://www.opengis.net/gml;>geometry24
 62 26 62 26 65 24 65 24 62=image/jpeg

http://localhost:8080/geoserver/eposgo/wms?service=WMS=1.3.0=GetMap=eposgo:MagnetotelluricStationsFeature=62,21,65,26=700=600=EPSG:4326=http://www.opengis.net/ogc; 
xmlns:gml="http://www.opengis.net/gml;>geometry62
 24 62 26 65 26 65 24 62 24=image/jpeg

WFS 2.0:

Query for all features works as expected:
localhost:8080/geoserver/eposgo/ows?service=WFS=2.0.0=GetFeature=eposgo:MagnetotelluricStationsFeature=application/json

maxFeatures does not work, the query returns all features:
localhost:8080/geoserver/eposgo/ows?service=WFS=2.0.0=GetFeature=eposgo:MagnetotelluricStationsFeature=10=application/json

Filtering the query by an inline bbox results in an  (from Geoserver log: 
"java.lang.RuntimeException: Unable to evaluate template path against the template"):
localhost:8080/geoserver/eposgo/ows?service=WFS=2.0.0=GetFeature=eposgo:MagnetotelluricStationsFeature=62,21,65,26,EPSG:4326=application/json

Spatial filtering the query by a CQL-BBOX does not work, it returns all 
features:
localhost:8080/geoserver/eposgo/ows?service=WFS=2.0.0=GetFeature=eposgo:MagnetotelluricStationsFeature=BBOX(62,21,65,26)=application/json

Spatial filtering the query with an OGC BOX or Contains returns  (java.lang.NullPointerException):
localhost:8080/geoserver/eposgo/ows?service=WFS=2.0.0=GetFeature=eposgo:MagnetotelluricStationsFeature=http://www.opengis.net/ogc; 
xmlns:gml="http://www.opengis.net/gml;>geometry21622665=application/json

Re: [Geoserver-users] "org.postgresql.util.PSQLException" in geoserver 2.21.2 (not in 2.21.1)

[Henning Lorenz]

Just to follow up with the reason and solution for the problem:

In an id-element in the app-schema mapping, I did not use the database 
id (integer primary key), but instead concatenated three columns to 
generate the in the output expected id of the feature (also unique), not 
expecting that this could generate problems. But it resulted in the 
offensive database-query. I removed the concatenation (and moved it to 
the features-templating) and everything works fine.


Thanks for all help with finding the problem,

Henning

On 2022-11-18 11:02, Hans Yperman wrote:

Page Title

I can shed some light on that as well: Click on 'blame' in the 
relevant git line, and it points to GEOT-7248


https://github.com/geotools/geotools/pull/4092

https://github.com/geotools/geotools/commit/855b5f4ae72ffa3117a94806888c647ef96c2cc2

This change was backported 11 days ago.  It seems to fix situations 
where there are 0 ID columns, but breaks everything where there are >1 
ID columns.  Your error talks about 3 ID columns, so you are impacted.


*Hans Yperman*

Department IT





*Vlaams Instituut voor de Zee vzw*

InnovOcean Campus, Jacobsenstraat 1

8400 Oostende, België

☎+32 (0) 59 33 61 13

hans.yper...@vliz.be <mailto:hans.yper...@vliz.be>

*www.vliz.be <http://www.vliz.be>***

*From:* Henning Lorenz 
*Sent:* vrijdag 18 november 2022 10:40
*To:* geoserver-users@lists.sourceforge.net
*Subject:* Re: [Geoserver-users] "org.postgresql.util.PSQLException" 
in geoserver 2.21.2 (not in 2.21.1)


Hi Hans, Julian,

Thank you, Hans, for the elaborate reply.

  * Activating "NumberMatched skip" as a workaround is successful
under geosever 2.21.2
  * Julian: the WFS query is

http://localhost:8080/geoserver/eposgo/ows?service=WFS=1.0.0=GetFeature=eposgo%3AMagnetotelluricStationsFeature=5=application%2Fjson

<http://localhost:8080/geoserver/eposgo/ows?service=WFS=1.0.0=GetFeature=eposgo%3AMagnetotelluricStationsFeature=5=application%2Fjson>
(this is copied from the layer preview and any format will
generate the error; same if I remove maxFeatures)
  * Although I have never touched "NumberMatched skip" before, I
checked the setting under geoserver 2.21.1, and it is as expected
deactivated. Thus,
  o the query above under geoserver 2.21.1 works totally fine
  o the query above under geoserver 2.21.2 works only with
"NumberMatched skip" activated

I have now a workaround and will try to follow up according to Hans's 
suggestion. But this still leaves two open questions:


  * Why has the behaviour of geoserver changed from 2.21.1 to 2.21.2?
  * Did the change introduce a bug, or is this expected behaviour?
(which would mean that the query worked under 2.21.1 because of a
bug that was eliminated in 2.21.2)

Best wishes,

Henning

On 2022-11-17 15:14, Hans Yperman wrote:

I'll add my 2 cents.

The problem is not with the driver, but with the query generated
by geoserver:  This text:

ERROR: function count(character varying, character varying,
character varying) does not exist Hint: No function matches the
given name and argument types. You might need to add explicit type
casts. Position: 8


is an SQL error message from postgres. The postgres jdbc driver is
just the transport of this message, not the cause.  The SQL is
generated by

at

org.geotools.appschema.jdbc.JoiningJDBCFeatureSource.getCountInternal(JoiningJDBCFeatureSource.java:1523)

so let's look at that

file:https://github.com/geotools/geotools/blob/main/modules/extension/app-schema/app-schema/src/main/java/org/geotools/appschema/jdbc/JoiningJDBCFeatureSource.java#L1523

The query comes from either createCountQuery or
createJoiningCountQuery, and both generate a query with this shape:

SELECT COUNT(DISTINCT idcolumn,idcolumn,…) FROM  (…)

Notice how COUNT is on position 8 in this query, which matches the
error message.  The error also claims there are 3 arguments given
to count.   Now, 3 args seems strange:  AFAIK the count syntax
accepts only 1 column.  If there are multiple ID columns,
geoserver does something I can't help you with.

Next steps could be:

1) Try to get hold of the actual query, e.g.  grab it from the
network with wireshark, log it on the postgres server, put a java
breakpoint on that line 1523,  …

2) Once you have it, run it on your database with pgadmin or psql
or …  .  Validate you get the same error message.  Play with it
until the error goes away.

3) Convince geoserver to generate a valid query based on whatever
you learned from 2).  I'd guess this would do something with the
id columns.

There might be a workaround:  Under Layer settings > Publish,
there is a setting that lets you skip the NumberMatched counting. 
If you can live without this count, you can hide the problem by

Re: [Geoserver-users] "org.postgresql.util.PSQLException" in geoserver 2.21.2 (not in 2.21.1)

[Henning Lorenz]

Ok, thank you for this information.

The immediate follow-up question is whether this is the end of 
app-schema feature-chaining? Where else do multiple id columns come from 
if not from several tables?(though I am not sure why I have 3 id columns 
when the service is only based on two tables that are 
"feature-chained"). I just hope that this is not the case.


Henning

On 2022-11-18 11:02, Hans Yperman wrote:

Page Title

I can shed some light on that as well: Click on 'blame' in the 
relevant git line, and it points to GEOT-7248


https://github.com/geotools/geotools/pull/4092

https://github.com/geotools/geotools/commit/855b5f4ae72ffa3117a94806888c647ef96c2cc2

This change was backported 11 days ago.  It seems to fix situations 
where there are 0 ID columns, but breaks everything where there are >1 
ID columns.  Your error talks about 3 ID columns, so you are impacted.


*Hans Yperman*

Department IT





*Vlaams Instituut voor de Zee vzw*

InnovOcean Campus, Jacobsenstraat 1

8400 Oostende, België

☎+32 (0) 59 33 61 13

hans.yper...@vliz.be <mailto:hans.yper...@vliz.be>

*www.vliz.be <http://www.vliz.be>***

*From:* Henning Lorenz 
*Sent:* vrijdag 18 november 2022 10:40
*To:* geoserver-users@lists.sourceforge.net
*Subject:* Re: [Geoserver-users] "org.postgresql.util.PSQLException" 
in geoserver 2.21.2 (not in 2.21.1)


Hi Hans, Julian,

Thank you, Hans, for the elaborate reply.

  * Activating "NumberMatched skip" as a workaround is successful
under geosever 2.21.2
  * Julian: the WFS query is

http://localhost:8080/geoserver/eposgo/ows?service=WFS=1.0.0=GetFeature=eposgo%3AMagnetotelluricStationsFeature=5=application%2Fjson

<http://localhost:8080/geoserver/eposgo/ows?service=WFS=1.0.0=GetFeature=eposgo%3AMagnetotelluricStationsFeature=5=application%2Fjson>
(this is copied from the layer preview and any format will
generate the error; same if I remove maxFeatures)
  * Although I have never touched "NumberMatched skip" before, I
checked the setting under geoserver 2.21.1, and it is as expected
deactivated. Thus,
  o the query above under geoserver 2.21.1 works totally fine
  o the query above under geoserver 2.21.2 works only with
"NumberMatched skip" activated

I have now a workaround and will try to follow up according to Hans's 
suggestion. But this still leaves two open questions:


  * Why has the behaviour of geoserver changed from 2.21.1 to 2.21.2?
  * Did the change introduce a bug, or is this expected behaviour?
(which would mean that the query worked under 2.21.1 because of a
bug that was eliminated in 2.21.2)

Best wishes,

Henning

On 2022-11-17 15:14, Hans Yperman wrote:

I'll add my 2 cents.

The problem is not with the driver, but with the query generated
by geoserver:  This text:

ERROR: function count(character varying, character varying,
character varying) does not exist Hint: No function matches the
given name and argument types. You might need to add explicit type
casts. Position: 8


is an SQL error message from postgres. The postgres jdbc driver is
just the transport of this message, not the cause.  The SQL is
generated by

at

org.geotools.appschema.jdbc.JoiningJDBCFeatureSource.getCountInternal(JoiningJDBCFeatureSource.java:1523)

so let's look at that

file:https://github.com/geotools/geotools/blob/main/modules/extension/app-schema/app-schema/src/main/java/org/geotools/appschema/jdbc/JoiningJDBCFeatureSource.java#L1523

The query comes from either createCountQuery or
createJoiningCountQuery, and both generate a query with this shape:

SELECT COUNT(DISTINCT idcolumn,idcolumn,…) FROM  (…)

Notice how COUNT is on position 8 in this query, which matches the
error message.  The error also claims there are 3 arguments given
to count.   Now, 3 args seems strange:  AFAIK the count syntax
accepts only 1 column.  If there are multiple ID columns,
geoserver does something I can't help you with.

Next steps could be:

1) Try to get hold of the actual query, e.g.  grab it from the
network with wireshark, log it on the postgres server, put a java
breakpoint on that line 1523,  …

2) Once you have it, run it on your database with pgadmin or psql
or …  .  Validate you get the same error message.  Play with it
until the error goes away.

3) Convince geoserver to generate a valid query based on whatever
you learned from 2).  I'd guess this would do something with the
id columns.

There might be a workaround:  Under Layer settings > Publish,
there is a setting that lets you skip the NumberMatched counting. 
If you can live without this count, you can hide the problem by
enabling that setting.

Good luck,

Hans

*Hans Yperman*

Department IT





*Vlaams Instituut voor de Ze

Re: [Geoserver-users] "org.postgresql.util.PSQLException" in geoserver 2.21.2 (not in 2.21.1)

[Henning Lorenz]

Hi Hans, Julian,

Thank you, Hans, for the elaborate reply.

 * Activating "NumberMatched skip" as a workaround is successful under
   geosever 2.21.2
 * Julian: the WFS query is
   
http://localhost:8080/geoserver/eposgo/ows?service=WFS=1.0.0=GetFeature=eposgo%3AMagnetotelluricStationsFeature=5=application%2Fjson
   (this is copied from the layer preview and any format will generate
   the error; same if I remove maxFeatures)
 * Although I have never touched "NumberMatched skip" before, I checked
   the setting under geoserver 2.21.1, and it is as expected
   deactivated. Thus,
 o the query above under geoserver 2.21.1 works totally fine
 o the query above under geoserver 2.21.2 works only with
   "NumberMatched skip" activated

I have now a workaround and will try to follow up according to Hans's 
suggestion. But this still leaves two open questions:


 * Why has the behaviour of geoserver changed from 2.21.1 to 2.21.2?
 * Did the change introduce a bug, or is this expected behaviour?
   (which would mean that the query worked under 2.21.1 because of a
   bug that was eliminated in 2.21.2)

Best wishes,

Henning


On 2022-11-17 15:14, Hans Yperman wrote:

Page Title

I'll add my 2 cents.

The problem is not with the driver, but with the query generated by 
geoserver:  This text:


ERROR: function count(character varying, character varying, character 
varying) does not exist Hint: No function matches the given name and 
argument types. You might need to add explicit type casts. Position: 8


is an SQL error message from postgres.  The postgres jdbc driver is 
just the transport of this message, not the cause.  The SQL is 
generated by


at 
org.geotools.appschema.jdbc.JoiningJDBCFeatureSource.getCountInternal(JoiningJDBCFeatureSource.java:1523)


so let's look at that 
file:https://github.com/geotools/geotools/blob/main/modules/extension/app-schema/app-schema/src/main/java/org/geotools/appschema/jdbc/JoiningJDBCFeatureSource.java#L1523


The query comes from either createCountQuery or 
createJoiningCountQuery, and both generate a query with this shape:


SELECT COUNT(DISTINCT idcolumn,idcolumn,…) FROM  (…)

Notice how COUNT is on position 8 in this query, which matches the 
error message.  The error also claims there are 3 arguments given to 
count.   Now, 3 args seems strange:  AFAIK the count syntax accepts 
only 1 column.  If there are multiple ID columns, geoserver does 
something I can't help you with.


Next steps could be:

1) Try to get hold of the actual query, e.g.  grab it from the network 
with wireshark, log it on the postgres server, put a java breakpoint 
on that line 1523,  …


2) Once you have it, run it on your database with pgadmin or psql or 
…  .  Validate you get the same error message.  Play with it until the 
error goes away.


3) Convince geoserver to generate a valid query based on whatever you 
learned from 2).  I'd guess this would do something with the id columns.


There might be a workaround:  Under Layer settings > Publish, there is 
a setting that lets you skip the NumberMatched counting.  If you can 
live without this count, you can hide the problem by enabling that 
setting.


Good luck,

Hans

*Hans Yperman*

Department IT





*Vlaams Instituut voor de Zee vzw*

InnovOcean Campus, Jacobsenstraat 1

8400 Oostende, België

☎+32 (0) 59 33 61 13

hans.yper...@vliz.be <mailto:hans.yper...@vliz.be>

*www.vliz.be <http://www.vliz.be>***

*From:* Henning Lorenz 
*Sent:* donderdag 17 november 2022 12:07
*To:* geoserver-users@lists.sourceforge.net
*Subject:* [Geoserver-users] "org.postgresql.util.PSQLException" in 
geoserver 2.21.2 (not in 2.21.1)


Hello,

I have a fully functional WFS service under geoserver 2.21.1 that 
results in a postgresql-related error in 2.22.2.


Setup: geoserver, extensions: app-schema, features-templating, 
ogc-api, smart-data-loader; PostgreSQL 15 (same for 14); jdk 11 (same 
for 17)


The service works under version 2.21.1 in Tomcat 9. The service fails 
under version 2.21.2 in Tomcat 9 and standalone with the following 
error (no other change to underlying software, configuration):


http://www.opengis.net/ogc
http://schemas.opengis.net/wfs/1.0.0/OGC-exception.xsd;

<http://www.opengis.net/ogchttp:/schemas.opengis.net/wfs/1.0.0/OGC-exception.xsd>>

java.lang.RuntimeException: java.io.IOException
java.io.IOExceptionERROR: function count(character varying,
character varying, character varying) does not exist Hint: No
function matches the given name and argument types. You might need
to add explicit type casts. Position: 8



According to the log, this is a "org.postgresql.util.PSQLException". I 
paste the complete entry from the log (geoserver developer logging) 
below. It appears to be a problem with the PostgreSQL driver, but I am 
not sure how to proceed and how to track the problem down, so 

[Geoserver-users] "org.postgresql.util.PSQLException" in geoserver 2.21.2 (not in 2.21.1)

[Henning Lorenz]

Hello,

I have a fully functional WFS service under geoserver 2.21.1 that results in a 
postgresql-related error in 2.22.2.

Setup: geoserver, extensions: app-schema, features-templating, ogc-api, 
smart-data-loader; PostgreSQL 15 (same for 14); jdk 11 (same for 17)

The service works under version 2.21.1 in Tomcat 9. The service fails under 
version 2.21.2 in Tomcat 9 and standalone with the following error (no other 
change to underlying software, configuration):

http://www.opengis.net/ogc 
http://schemas.opengis.net/wfs/1.0.0/OGC-exception.xsd;>

java.lang.RuntimeException: java.io.IOException java.io.IOExceptionERROR: 
function count(character varying, character varying, character varying) does 
not exist Hint: No function matches the given name and argument types. You 
might need to add explicit type casts. Position: 8


According to the log, this is a "org.postgresql.util.PSQLException". I paste 
the complete entry from the log (geoserver developer logging) below. It appears to be a 
problem with the PostgreSQL driver, but I am not sure how to proceed and how to track the 
problem down, so any advice is highly appreciated. (Was the driver changed during 
2.21.1/2 development? Is it possible to downgrade the driver and test?)

Henning

ERROR  [geoserver.ows] -
java.lang.RuntimeException: java.io.IOException
   at 
org.geotools.data.complex.MappingFeatureCollection.size(MappingFeatureCollection.java:329)
   at org.geoserver.wfs.CountExecutor.getCount(CountExecutor.java:44)
   at org.geoserver.wfs.GetFeature.getTotalCount(GetFeature.java:912)
   at org.geoserver.wfs.GetFeature.updateTotalCount(GetFeature.java:725)
   at org.geoserver.wfs.GetFeature.run(GetFeature.java:637)
   at 
org.geoserver.wfs.DefaultWebFeatureService.getFeature(DefaultWebFeatureService.java:109)
   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
   at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.base/java.lang.reflect.Method.invoke(Method.java:568)
   at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
   at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
   at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
   at 
org.geoserver.ows.util.RequestObjectLogger.invoke(RequestObjectLogger.java:51)
   at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
   at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
   at jdk.proxy3/jdk.proxy3.$Proxy126.getFeature(Unknown Source)
   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
   at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.base/java.lang.reflect.Method.invoke(Method.java:568)
   at org.geoserver.ows.Dispatcher.execute(Dispatcher.java:867)
   at 
org.geoserver.ows.Dispatcher.handleRequestInternal(Dispatcher.java:268)
   at 
org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:177)
   at 
org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:52)
   at 
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1043)
   at 
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
   at 
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
   at 
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
   at 
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
   at 
org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459)
   at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
   at 
org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
   at 
org.geoserver.filters.ThreadLocalsCleanupFilter.doFilter(ThreadLocalsCleanupFilter.java:28)
   at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
   at 
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
   at 

Re: [Geoserver-users] validation fails: randomness required for gml:id/idExpression of Application Schema while joining support is active

[Henning Lorenz]

Dear Marco,

Thank you very much for your suggestions! It is not the solution, but 
the right direction. I tried a couple of things and, eventually, found a 
solution that indeed is related to primary keys/database indices. For 
the record, in case others stumble over the same problem:


 * In the mapping file, primary keys were exposed
 * I used database views to flatten the structure of some features
   (i.e. now primary key or unique index possible)
 * This setup worked with older geoserver versions (2.14 - 2.16, I guess)

Since then, something must have changed "under the hood", which resulted 
in the described problem and also some other issues that were difficult 
to pin down exactly.


 * I moved all normal database views to materialized views
 * This allows the generation of indices, and I generated a unique
   index on each of the materialized views

When this was done, all issues disappeared immediately and the App 
Schema based service worked again as before.


Best regards,

Henning



On 2022-10-07 14:57, Marco Volpini wrote:

Dear Henning,
you might try if setting the expose primary key flag to false, inside 
the datastore params in your app-schema mappings fixes the issue:


   Expose primary keys
   false

I'm not sure 100% sure it will work for your use case, but in this way 
GeoServer will generate random UUID for features FID instead of using 
the database primary key and if the same instance is not reused across 
feature chaining should lead to the desired result (however not 
exposing pk will probably cause performance degradation when querying 
the data).
If this does not work then I believe that code changes after a careful 
code investigation are needed in order to support your use case.


Kind Regards,

Marco Volpini

==GeoServer Professional Services from the experts!

Visit http://bit.ly/gs-services-us <http://bit.ly/gs-services-us>for 
more information.==Marco Volpini


Software EngineerGeoSolutions Groupphone: +39 0584 962313

fax:     +39 0584 1660272


https://www.geosolutionsgroup.com/ <https://www.geosolutionsgroup.com/>

http://twitter.com/geosolutions_it <http://twitter.com/geosolutions_it>

---


Con riferimento alla normativa sul trattamento dei dati personali 
(Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati 
“GDPR”), si precisa che ogni circostanza inerente alla presente email 
(il suo contenuto, gli eventuali allegati, etc.) è un dato la cui 
conoscenza è riservata al/i solo/i destinatario/i indicati dallo 
scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a 
cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato 
se potesse darmene notizia.This email is intended only for the person 
or entity to which it is addressed and may contain information that is 
privileged, confidential or otherwise protected from disclosure. We 
remind that - as provided by European Regulation 2016/679 “GDPR” - 
copying, dissemination or use of this e-mail or the information herein 
by anyone other than the intended recipient is prohibited. If you have 
received this email by mistake, please notify us immediately by 
telephone or e-mail.





On Thu, Oct 6, 2022 at 9:31 AM Henning Lorenz 
 wrote:


geoserver 2.21.1, Postgresql

Hello,

Question: How can I add randomness to App Schema idExpression
while joining support is active? (then random() is not supported)

Background:

  * App Schema automatically encodes the ids based on
..
  * Ids have to be unique within the output document, else the
document will not validate.

If a mapping file repeatedly chains the same feature and there the
same content (same line in the table, same primary key), the
generated id is the same. (For me this happens with a mapping file
that chains responsible party (gmd:CI_ResponsibleParty) in several
places. When the same party is responsible more than once, the
generated ids are the same and thus, the output document does not
validate).

As all data are the same in these multiple occurrences, the only
solution I see is to add a random string/number. To concatenate
random() in the idExpression is a solution in case joining support
is deactivated. However, with activated joining support, it
generates an error (only database functions are supported, as
indicated in the documentation). But when I turn of joining
support, JDBC multi values support seems to be deactivated as
well. Thus, this is not a solution. Therefore, I need a solution
for adding randomness while joining support is active.

Thanks in advance for any help on this topic,

Henning

The funny thing is that the same app schema mapping with random()
and multiple values did work and produce valid output on a now
decommissioned server. The geoserver version was in the range
2.14-2.16, and I can't re

[Geoserver-users] validation fails: randomness required for gml:id/idExpression of Application Schema while joining support is active

[Henning Lorenz]

geoserver 2.21.1, Postgresql

Hello,

Question: How can I add randomness to App Schema idExpression while joining 
support is active? (then random() is not supported)

Background:

 *   App Schema automatically encodes the ids based on ..
 *   Ids have to be unique within the output document, else the document will 
not validate.

If a mapping file repeatedly chains the same feature and there the same content 
(same line in the table, same primary key), the generated id is the same. (For 
me this happens with a mapping file that chains responsible party 
(gmd:CI_ResponsibleParty) in several places. When the same party is responsible 
more than once, the generated ids are the same and thus, the output document 
does not validate).

As all data are the same in these multiple occurrences, the only solution I see 
is to add a random string/number. To concatenate random() in the idExpression 
is a solution in case joining support is deactivated. However, with activated 
joining support, it generates an error (only database functions are supported, 
as indicated in the documentation). But when I turn of joining support, JDBC 
multi values support seems to be deactivated as well. Thus, this is not a 
solution. Therefore, I need a solution for adding randomness while joining 
support is active.

Thanks in advance for any help on this topic,

Henning

The funny thing is that the same app schema mapping with random() and multiple 
values did work and produce valid output on a now decommissioned server. The 
geoserver version was in the range 2.14-2.16, and I can't recall whether 
joining support was active. Maybe it just was a coincidence, or something has 
changed in geoserver.








När du har kontakt med oss på Uppsala universitet med e-post så innebär det att 
vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du 
läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/

E-mailing Uppsala University means that we will process your personal data. For 
more information on how this is performed, please read here: 
http://www.uu.se/en/about-uu/data-protection-policy
___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

[Geoserver-users] App Schema for attributes with multiple values and nested attributes

[Henning Lorenz]

Title: Page Title



Geoserver 2.21.1, AppSchema, PostGIS datastore

I have problems to implement attributes that have multiple values and nested attributes, and which are not a feature type. As long as I stick strictly to the "attributes with multiple values" examples in the documentation, everything works fine (which suggests
 that my setup works). But when I try to adapt the output to what is required for the example of swe:DataRecord (below), I either get only one attribute in return or none at all. Below is what I want to achieve and one of my attempts. Is it possible at all?
 And if yes, what is the correct approach?
Thanks in advance for any help.
Henning
--

I have two tables in my PostGIS database:

geologylogs contains information and data of individual logs geologylogfields contains the fields that are present in each of the logs (1:n relationship to geologylogs via FK):






id (PK)

logid (FK to geologylogs)

name (name of the field, e.g. 'lithology', 'colour', ...)
definition (short explanation of the name)
url (points to vocabulary entry for name)


1

log1

nameA

definitionA

urlA



2

log1

nameB

definitionB

urlB



3

log1

nameC

definitionC

urlC



...















This should be mapped to the following output, where swe:DataRecord is the feature (source table geologylogfields) that is (successfully) chained from gwml2w:GW_GeologyLog (source table geologylogs). Each swe:DataRecord has one to many occurrences of swe:field
 (like feature chaining, but without the feature type):

    
    nameA">
    
    definitionA
    
    
    
    
    definitionB
    
    
    
    
    definitionB
    
    
And the mapping file of one attempt, which produces a total of one  instead of one  for each entry that is related to the parent log. If I only use the AttributeMapping with the isMultiple true statement, I don't get a swe:field at
 all (probably because not all required elements are present). 

    
    datastore
    geologylogfields
    swe:DataRecord
    
    
    FEATURE_LINK
    
    logid
    
    
    
    swe:DataRecord
    
    Concatenate('SD-', logid, '-geollog-datarecord')
    
    
    definition
    'http://www.opengis.net/def/gwml/2.0/coverage/geologyLog'
    
    

    
    swe:field
    
    name
    name
    
    true
    
    
    swe:field/swe:Category
    
    definition
    url
    
    
    
    swe:field/swe:Category/swe:description
    
    definition
    
    












När du har kontakt med oss på Uppsala universitet med e-post så innebär det att vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/


E-mailing Uppsala University means that we will process your personal data. For more information on how this is performed, please read here: http://www.uu.se/en/about-uu/data-protection-policy



___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Smart Data Loader and Feature Templating plugins fail to work with Array Type (PostgreSQL)

[Henning Lorenz]

Hi Andrea,

Thanks for your thoughts and background information. After some testing, 
I can confirm that Array Type columns from the PostGIS data source are 
handled as expected (geoserver 2.21.1). But it wasn't clear to me from 
the documentation that this is only possible when directly feature 
templating a layer of a PostGIS data source. Just to sum up the present 
capabilities:


 * Layer based on PostGIS datastore: Array Type data are handled
   correctly by Feature Templating, no access to related tables
 * Layer based on App Schema datastore: Related tables can be accessed
   by Feature Templating, but Array Type data are not handled (error)

Let's see how this develops. Thanks again Nuno & Andrea for answering my 
questions.


Best, Henning


On 2022-09-13 09:38, Andrea Aime wrote:

Hi,
array support is working properly in the context of the STAC API usage 
of features templating, which in turn
uses a customized data store producing, among others, properties of 
type array, and then feature templates are applied

to generate STAC Items.
This is on the main branch, although some level of support is 
available on 2.21.x too (don't remember exactly what got backported 
and what did not).


In your case, what are you doing? Using features templating directly 
against a PostGIS data source, to customize the output
of a WFS or OGC API Features? I don't think we ever tried that, it may 
work, or not.


In terms of getting it to work, you'll have to find either programming 
time or a bag of money, or wait for someone else to stumble
into the same problem, and act on it. If you want to investigate the 
code on your side and have questions, please ask on the list,
always happy to get another developer up to speed (within reason of 
course, all the activity on this list is volunteer based).


Cheers
Andrea


On Tue, Sep 13, 2022 at 8:55 AM Henning Lorenz 
 wrote:


Hi Nuno,

Thinking about your reply, I can't bring it in line with the
documentation and my observations, because I don't create array
attributes with App Schema:

  * A column with Array Type is defined in PostgreSQL, i.e. the
database query returns an array to Geoserver
  * The Feature Templating documentation that I linked earlier
clearly refers to Array Type in databases (i.e. the situation
in my case, if I do not completely misunderstand)
  * Feature Templating produces a "[Ljava.lang.String;@..."
<mailto:[Ljava.lang.String;@...> error
  * My web searches suggest that a "[Ljava.lang.String;@..."
<mailto:[Ljava.lang.String;@...> error indicates that an array
is passed to a function that expects string only, which
suggests that App Schema correctly passes on the Array Type
data to Feature Templating, but Feature templating doesn't
understand it in contradiction to the documentation.

A first step would be to decide whether I am unable to interpret
the documentation correctly. Then to think what resources are
needed for full array support and how to get them (I'd like to
have a bag of money for this, but I don't ...).

Best wishes,

Henning


On 2022-09-13 00:11, Nuno Oliveira wrote:

Hi Henning,
that's a good point, if my memory serves me well, if an array
reaches features-templating it will work as described (this is
used for some advanced integrations with features-templating),
but I don't think that App-Schema and Smart Data Loader can
produce array attributes.

So summarizing:

  * Features-templating support arrays,
  * but you cannot produce array attributes with App-Schema or
Smart Data Loader,
  * hence you can't use arrays unless you create them
programmatically.

@Marco Volpini <mailto:marco.volp...@geosolutionsgroup.com> did I
miss something?

Kind regards,
Nuno Oliveira




    On Fri, Sep 9, 2022 at 12:27 PM Henning Lorenz
 wrote:

Dear Nuno,

Thanks for the quick reply. However, for the Feature
Templating plugin, it contradicts the documentation where
Array Type support for json output is explicitly mentioned:

https://docs.geoserver.org/stable/en/user/community/features-templating/directives.html#array-based-properties-json-based-templates-only

Is the documentation wrong and needs to be updated?

Best wishes,

Henning


On 2022-09-09 11:42, Nuno Oliveira wrote:

Dear Henning,
those modules do not support array column types, this would
be a nice feature to have.

A pull request contributing it would be very welcomed, you
may also want to check the GeoServer Support
<https://geoserver.org/support/> in case you want to fund it.

Kind regards,
Nuno Oliveira

    On Fri, Sep 9, 2022 at 10:13 AM Henning Lorenz
 wrote:

Re: [Geoserver-users] Smart Data Loader and Feature Templating plugins fail to work with Array Type (PostgreSQL)

[Henning Lorenz]

Hi Nuno,

Thinking about your reply, I can't bring it in line with the 
documentation and my observations, because I don't create array 
attributes with App Schema:


 * A column with Array Type is defined in PostgreSQL, i.e. the database
   query returns an array to Geoserver
 * The Feature Templating documentation that I linked earlier clearly
   refers to Array Type in databases (i.e. the situation in my case, if
   I do not completely misunderstand)
 * Feature Templating produces a "[Ljava.lang.String;@..." error
 * My web searches suggest that a "[Ljava.lang.String;@..." error
   indicates that an array is passed to a function that expects string
   only, which suggests that App Schema correctly passes on the Array
   Type data to Feature Templating, but Feature templating doesn't
   understand it in contradiction to the documentation.

A first step would be to decide whether I am unable to interpret the 
documentation correctly. Then to think what resources are needed for 
full array support and how to get them (I'd like to have a bag of money 
for this, but I don't ...).


Best wishes,

Henning


On 2022-09-13 00:11, Nuno Oliveira wrote:

Hi Henning,
that's a good point, if my memory serves me well, if an array reaches 
features-templating it will work as described (this is used for some 
advanced integrations with features-templating), but I don't think 
that App-Schema and Smart Data Loader can produce array attributes.


So summarizing:

  * Features-templating support arrays,
  * but you cannot produce array attributes with App-Schema or Smart
Data Loader,
  * hence you can't use arrays unless you create them programmatically.

@Marco Volpini <mailto:marco.volp...@geosolutionsgroup.com> did I miss 
something?


Kind regards,
Nuno Oliveira




On Fri, Sep 9, 2022 at 12:27 PM Henning Lorenz 
 wrote:


Dear Nuno,

Thanks for the quick reply. However, for the Feature Templating
plugin, it contradicts the documentation where Array Type support
for json output is explicitly mentioned:

https://docs.geoserver.org/stable/en/user/community/features-templating/directives.html#array-based-properties-json-based-templates-only

Is the documentation wrong and needs to be updated?

Best wishes,

Henning


On 2022-09-09 11:42, Nuno Oliveira wrote:

Dear Henning,
those modules do not support array column types, this would be a
nice feature to have.

A pull request contributing it would be very welcomed, you may
also want to check the GeoServer Support
<https://geoserver.org/support/> in case you want to fund it.

Kind regards,
Nuno Oliveira

On Fri, Sep 9, 2022 at 10:13 AM Henning Lorenz
 wrote:

geoserver 2.21, app schema/smart data loader/ogc api/feature
templating plugins (Tomcat 9 on Ubuntu 22.04; openjdk 11.0.16)

Hello,

I appreciate any help on the following two (related?)
problems (further explained below):

 1. The "Smart Data Loader" plugin fails in case Array Type
is present in one of the tables (error unknown data type)
 2. The "Feature Templating" plugin does not handle data from
columns with Array Type correctly
("[Ljava.lang.String;@..." <mailto:[Ljava.lang.String;@...>)

Of particular importance to me is to solve problem 2, as this
directly affects geoserver output.

Many thanks in advance,

Henning

-

Problem 1:

  * I create two tables, one without and one without a column
of Array Type (called here "noarray_table" and
"array_table"; only for explanation instead of my more
complex setup):

CREATE TABLE test.noarray_table (
    id SERIAL PRIMARY KEY,
    geometry geometry(point, 4326),
    attribute1 varchar,
    attribute2 varchar
);

CREATE TABLE test.array_table (
    id SERIAL PRIMARY KEY,
    geometry geometry(point, 4326),
    attribute1 varchar,
    attribute2 varchar[]
);

  * After adding some data, I create a PostGIS data store for
database that contains the tables.
  * When this is done, I add a new data store and select
"Smart Data Loader".
  o When I select the noarray_table as root entity, I get
the expected preview and can add the data store.
  o However, when I select the array_table, I get an
error message over the entire page. As far as I can
tell, the critical line is "Caused by:
java.lang.RuntimeException: Attribute type '_varcha

Re: [Geoserver-users] Smart Data Loader and Feature Templating plugins fail to work with Array Type (PostgreSQL)

[Henning Lorenz]

Dear Nuno,

Thanks for the quick reply. However, for the Feature Templating plugin, 
it contradicts the documentation where Array Type support for json 
output is explicitly mentioned: 
https://docs.geoserver.org/stable/en/user/community/features-templating/directives.html#array-based-properties-json-based-templates-only


Is the documentation wrong and needs to be updated?

Best wishes,

Henning


On 2022-09-09 11:42, Nuno Oliveira wrote:

Dear Henning,
those modules do not support array column types, this would be a nice 
feature to have.


A pull request contributing it would be very welcomed, you may also 
want to check the GeoServer Support <https://geoserver.org/support/> 
in case you want to fund it.


Kind regards,
Nuno Oliveira

On Fri, Sep 9, 2022 at 10:13 AM Henning Lorenz 
 wrote:


geoserver 2.21, app schema/smart data loader/ogc api/feature
templating plugins (Tomcat 9 on Ubuntu 22.04; openjdk 11.0.16)

Hello,

I appreciate any help on the following two (related?) problems
(further explained below):

 1. The "Smart Data Loader" plugin fails in case Array Type is
present in one of the tables (error unknown data type)
 2. The "Feature Templating" plugin does not handle data from
columns with Array Type correctly ("[Ljava.lang.String;@..."
<mailto:[Ljava.lang.String;@...>)

Of particular importance to me is to solve problem 2, as this
directly affects geoserver output.

Many thanks in advance,

Henning

-

Problem 1:

  * I create two tables, one without and one without a column of
Array Type (called here "noarray_table" and "array_table";
only for explanation instead of my more complex setup):

CREATE TABLE test.noarray_table (
    id SERIAL PRIMARY KEY,
    geometry geometry(point, 4326),
    attribute1 varchar,
    attribute2 varchar
);

CREATE TABLE test.array_table (
    id SERIAL PRIMARY KEY,
    geometry geometry(point, 4326),
    attribute1 varchar,
    attribute2 varchar[]
);

  * After adding some data, I create a PostGIS data store for
database that contains the tables.
  * When this is done, I add a new data store and select "Smart
Data Loader".
  o When I select the noarray_table as root entity, I get the
expected preview and can add the data store.
  o However, when I select the array_table, I get an error
message over the entire page. As far as I can tell, the
critical line is "Caused by: java.lang.RuntimeException:
Attribute type '_varchar' is unknown.". _varchar is the
same as varchar[] as far as I can tell.

Is this a bug or does Smart Data Loader not support Array Types
("missing feature")?


Problem 2:

The Feature Templating plugin explicitly supports Array Types
(see

https://docs.geoserver.org/stable/en/user/community/features-templating/directives.html#array-based-properties-json-based-templates-only)

  * With the Smart Data Loader, I create an app schema mapping and
schema of tables without Array Type
  * I configure feature templating and I get valid output.
  * However, one of the columns should actually be of array type.
Thus, I replace this varchar-column by a varchar[]-column,
including type casting of the content.
  o According to the documentation, I expect that the Array
Type of the column is recognised by the plugin and the
data are expanded accordingly.
  o However, I see "[Ljava.lang.String;@..."
<mailto:[Ljava.lang.String;@...> instead of data.
According to what I could find out, this indicates the
presence of array data where no array data are expected.

This leaves two options:

A) It is not sufficient to just change the Data Type in the
database, something in the configuration has to be changed as well
(App Schema mapping file, schema - although Array Type is not
supported for XML according to the documentation, which would
suggest that this is not the case).

B) The Feature Templating plugin doesn't work as expected, i.e.
it's a bug.










När du har kontakt med oss på Uppsala universitet med e-post så
innebär det att vi behandlar dina personuppgifter. För att läsa
mer om hur vi gör det kan du läsa här:
http://www.uu.se/om-uu/dataskydd-personuppgifter/

E-mailing Uppsala University means that we will process your
personal data. For more information on how this is performed,
please read here: http://www.uu.se/en/about-uu/data-protection-policy
_

[Geoserver-users] Smart Data Loader and Feature Templating plugins fail to work with Array Type (PostgreSQL)

[Henning Lorenz]

geoserver 2.21, app schema/smart data loader/ogc api/feature templating plugins 
(Tomcat 9 on Ubuntu 22.04; openjdk 11.0.16)

Hello,

I appreciate any help on the following two (related?) problems (further 
explained below):

 1.  The "Smart Data Loader" plugin fails in case Array Type is present in one 
of the tables (error unknown data type)
 2.  The "Feature Templating" plugin does not handle data from columns with Array Type 
correctly ("[Ljava.lang.String;@...")

Of particular importance to me is to solve problem 2, as this directly affects 
geoserver output.

Many thanks in advance,

Henning

-

Problem 1:

 *   I create two tables, one without and one without a column of Array Type (called here 
"noarray_table" and "array_table"; only for explanation instead of my more 
complex setup):

CREATE TABLE test.noarray_table (
   id SERIAL PRIMARY KEY,
   geometry geometry(point, 4326),
   attribute1 varchar,
   attribute2 varchar
);

CREATE TABLE test.array_table (
   id SERIAL PRIMARY KEY,
   geometry geometry(point, 4326),
   attribute1 varchar,
   attribute2 varchar[]
);

 *   After adding some data, I create a PostGIS data store for database that 
contains the tables.
 *   When this is done, I add a new data store and select "Smart Data Loader".
*   When I select the noarray_table as root entity, I get the expected 
preview and can add the data store.
*   However, when I select the array_table, I get an error message over the entire 
page. As far as I can tell, the critical line is "Caused by: 
java.lang.RuntimeException: Attribute type '_varchar' is unknown.". _varchar is the 
same as varchar[] as far as I can tell.

Is this a bug or does Smart Data Loader not support Array Types ("missing 
feature")?


Problem 2:

The Feature Templating plugin explicitly supports Array Types
(see 
https://docs.geoserver.org/stable/en/user/community/features-templating/directives.html#array-based-properties-json-based-templates-only)

 *   With the Smart Data Loader, I create an app schema mapping and schema of 
tables without Array Type
 *   I configure feature templating and I get valid output.
 *   However, one of the columns should actually be of array type. Thus, I 
replace this varchar-column by a varchar[]-column, including type casting of 
the content.
*   According to the documentation, I expect that the Array Type of the 
column is recognised by the plugin and the data are expanded accordingly.
*   However, I see 
"[Ljava.lang.String;@..." instead of data. 
According to what I could find out, this indicates the presence of array data where no array 
data are expected.

This leaves two options:

A) It is not sufficient to just change the Data Type in the database, something 
in the configuration has to be changed as well (App Schema mapping file, schema 
- although Array Type is not supported for XML according to the documentation, 
which would suggest that this is not the case).

B) The Feature Templating plugin doesn't work as expected, i.e. it's a bug.









När du har kontakt med oss på Uppsala universitet med e-post så innebär det att 
vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du 
läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/

E-mailing Uppsala University means that we will process your personal data. For 
more information on how this is performed, please read here: 
http://www.uu.se/en/about-uu/data-protection-policy
___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Limit namespaces in a WFS response to the ones required by the schema

[Henning Lorenz]

Thanks, Andrea.

If it is not easy to get the used namespaces from the application, a
different solution would be to restrict the namespaces in the WFS
response to the ones declared in the namespaces tag of app-schema
mapping file. Then its up to user to get it right - which is required in
any case if the WFS response should validate against the schema. But I
understand that this is presently not an option and way beyond what I
could achieve. So I'll look into setting up a second geoserver to
separate the namespaces.

Best,

Henning

> Date: Thu, 9 May 2019 17:31:34 +0200
> From: Andrea Aime 
> To: Henning Lorenz 
> Cc: GeoServer Mailing List List
>   
> Subject: Re: [Geoserver-users] Limit namespaces in a WFS response to
>   the ones required by the schema
> Message-ID:
>   
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
> I'm afraid not, it's not possible right now. With a code change it should
> be possible, for simple features it should not be
> too hard, for complex features, pretty hard (one would have to figure out
> which namespaces are actually used inside the complex model,
> which is an information available, but not easy to get out).
>
> Cheers
> Andrea
>
> On Thu, May 9, 2019 at 4:50 PM Henning Lorenz 
> wrote:
>
>> Geoserver 2.15.0
>>
>> Hello,
>> Is it possible to limit the listed namespaces in a WFS response produced
>> by geoserver to the ones required by the schema?
>>
>> Setting:
>> In the concerned geoserver installation, the data/workspaces directory
>> contains workspace directories of different purpose:
>> a) one workspace directory that provides WFS layers via app-schema
>> plugin, i.e. represents the primary namespace
>> b) a number of workspace directories with the sole purpose of providing
>> the namespaces required by the schema of a)
>> c) a number of workspace directory that are not related to a) and b),
>> e.g. host WMS layers.
>>
>> Problem:
>> The WFS response on a request for a) contains a list of a) + b) + c) in
>> the xmlns section:
>> 
>> >   xmlns:wfs="http://www.opengis.net/wfs/2.0;
>>   xmlns:xs="http://www.w3.org/2001/XMLSchema;
>>   xmlns - a)
>>   xmlns - all of b)
>>   xmlns - all of c)
>>
>> Is there a way to get rid of the unrelated and unwanted workspaces (i.e.
>> c) ) in the xmlns list? I.e. that the WFS response only contains the
>> namespaces required by the schema and not the entire list of workspaces
>> in the geoserver installation?
>>
>> Many thanks for any advice,
>>
>> Henning
>>
>>




smime.p7s
Description: S/MIME Cryptographic Signature
___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

[Geoserver-users] Limit namespaces in a WFS response to the ones required by the schema

[Henning Lorenz]

Geoserver 2.15.0

Hello,
Is it possible to limit the listed namespaces in a WFS response produced
by geoserver to the ones required by the schema?

Setting:
In the concerned geoserver installation, the data/workspaces directory
contains workspace directories of different purpose:
a) one workspace directory that provides WFS layers via app-schema
plugin, i.e. represents the primary namespace
b) a number of workspace directories with the sole purpose of providing
the namespaces required by the schema of a)
c) a number of workspace directory that are not related to a) and b),
e.g. host WMS layers.

Problem:
The WFS response on a request for a) contains a list of a) + b) + c) in
the xmlns section:

http://www.opengis.net/wfs/2.0;
  xmlns:xs="http://www.w3.org/2001/XMLSchema;
  xmlns - a)
  xmlns - all of b)
  xmlns - all of c)

Is there a way to get rid of the unrelated and unwanted workspaces (i.e.
c) ) in the xmlns list? I.e. that the WFS response only contains the
namespaces required by the schema and not the entire list of workspaces
in the geoserver installation?

Many thanks for any advice,

Henning





smime.p7s
Description: S/MIME Cryptographic Signature
___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] CSW plugin prevents start of Geoserver

[Henning Lorenz]

Hi Andrea,

Thanks for a quick answer.
No there is nothing in the log. Actually, the last entries are from the upgrade 
2.9.0=>2.9.1, i.e. from the installation of 2.9.1 (always overwrites the entire 
geoserver directory).
I tried different logging profiles - no success. And these changes to the 
logging settings do not persist after a restart of Geoserver. Then logging is 
always back to DEFAULT.
Maybe I should install the universal jetty-based binary and try again.

Cheers,

Henning


On 19 Aug 2016, at 16:01, Andrea Aime 
<andrea.a...@geo-solutions.it<mailto:andrea.a...@geo-solutions.it>> wrote:

Hi Henning,
no, this is new to me, we actually have one demo server using a 2.9.x series 
deploying
CSW and starting without problems here:
http://demo.geo-solutions.it/geoserver/web/

The server deploys once a day the current nightly build, and did not exhibit 
any startup problem around the
time 2.9.1 was released (a release is pretty much the same as a nightly, minus 
the installers
that make it easier to setup).

Can you check the GeoServer logs and see if you can find significant (or any) 
stack traces?

Cheers
Andrea


On Fri, Aug 19, 2016 at 3:21 PM, Henning Lorenz 
<henning.lor...@geo.uu.se<mailto:henning.lor...@geo.uu.se>> wrote:
Geoserver 2.9.1 web archive, tomcat 8.0.35, java 1.8.0_91, Ubuntu 16.04

Hello,

- Geoserver 2.9.1 works fine without plugin
- Geoserver 2.9.1 does not start with the csw 2.9.1 plugin installed
- the problematic jars are gs-csw-core-2.9.1.jar and gs-web-csw-2.9.1.jar: 
Geoserver does not start if any of the two jars is in WEB-INF/lib, the other 
jars of the CSW plugin do not cause any trouble.

Has anybody else observed this problem and are there any solutions?
Thanks for helping,

Henning
--
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net<mailto:Geoserver-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/geoserver-users



--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it<http://www.geo-solutions.it/>
http://twitter.com/geosolutions_it


AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i 
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo 
è consentito esclusivamente al destinatario del messaggio, per le finalità 
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne 
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di 
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro 
sistema. Conservare il messaggio stesso, divulgarlo anche in parte, 
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, 
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.



The information in this message and/or attachments, is intended solely for the 
attention and use of the named addressee(s) and may be confidential or 
proprietary in nature or covered by the provisions of privacy act (Legislative 
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in 
accord with its purpose, any disclosure, reproduction, copying, distribution, 
or either dissemination, either whole or partial, is strictly forbidden except 
previous formal approval of the named addressee(s). If you are not the intended 
recipient, please contact immediately the sender by telephone, fax or e-mail 
and delete the information in this message that has been received in error. The 
sender does not give any warranty or accept liability as the content, accuracy 
or completeness of sent messages and accepts no responsibility  for changes 
made after they were sent or for other risks which arise as a result of e-mail 
transmission, viruses, etc.

---

--
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

[Geoserver-users] CSW plugin prevents start of Geoserver

[Henning Lorenz]

Geoserver 2.9.1 web archive, tomcat 8.0.35, java 1.8.0_91, Ubuntu 16.04

Hello,

- Geoserver 2.9.1 works fine without plugin
- Geoserver 2.9.1 does not start with the csw 2.9.1 plugin installed
- the problematic jars are gs-csw-core-2.9.1.jar and gs-web-csw-2.9.1.jar: 
Geoserver does not start if any of the two jars is in WEB-INF/lib, the other 
jars of the CSW plugin do not cause any trouble.

Has anybody else observed this problem and are there any solutions?
Thanks for helping,

Henning
--
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

[Geoserver-users] WMS request returns empty layer list

[Henning Lorenz]

Geoserver 2.7.4, Tomcat7, Ubuntu 14, Java 1.7

Hello,

The setup is very simple:
1 Workspace
1 Store: A PostGis table with point geometry
1 Layer from the above Store, enabled and advertised
WMS Services are global only and activated.
Layer preview in Geoserver-OpenLayers is fine.
But when I try to access the layer in a client, then the layer list is empty. I 
tried to connect to Geoserver from both QGis and Mapbender - same result: The 
connection to Geoserver works but the list of layers that is returned to the 
client by Geoserver is empty, although the layer is enabled and advertised in 
the layer settings. QGis and Mapbender work fine with other servers, so the 
problem is with Geoserver.

The same problem occurred also with a shape-file store in an earlier attempt - 
I started over from scratch after repeated failures to get this to work by 
following the simple tutorials in the documentation step-by-step, without 
success.
Any help is very appreciated, thank you.

Henning



--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users