Andrea,
First, I forgot to say this in on GeoServer 2.6.2, postgres 9.3.6, Linux
Ubuntu 14.04. JNDI is NOT in use at this point. Just 2 mosaics on this
machine. Nothing else defined or in use.
The error IS connection related ... which is what I think you were asking
related to leaking connections from the pool. I am seeing an error saying
that a connection can't be established ... "FATAL: remaining connection
slots are reserved for non-replication superuser connections" ... so the
connections are exhausted. Partial stack trace is below, and the very base
of the error is in getConnection() ... so the connection can't be made.
ERROR [gce.imagemosaic] -
java.io.IOException
at
org.geotools.gce.imagemosaic.CatalogManager.createGranuleCatalogFromDatastore(CatalogManage
r.java:200)
... [omitting full trace - can send if you need it] ...
... 103 more
Caused by: org.postgresql.util.PSQLException: FATAL: remaining connection
slots are reserved for non-replication superuser connections
at
org.postgresql.core.v3.ConnectionFactoryImpl.readStartupMessages(ConnectionFactoryImpl.java:464)
at
org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:112)
at
org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
at
org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
at
org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
at
org.postgresql.jdbc3.Jdbc3Connection.<init>(Jdbc3Connection.java:24)
at org.postgresql.Driver.makeConnection(Driver.java:393)
at org.postgresql.Driver.connect(Driver.java:267)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at
org.geotools.data.postgis.PostgisNGDataStoreFactory.getConnection(PostgisNGDataStoreFactory.java:272)
at
org.geotools.data.postgis.PostgisNGDataStoreFactory.createDataSource(PostgisNGDataStoreFactory.java:222)
... 106 more
As far as the mosaics ... as I mentioned, this is just for 2 image
mosaics. There is nothing else on this box at this point (except one
vector store with one layer that's not being used). I thought the default
max connections in each pool was for 10 per store, but have experimented
with max\ connections set to 5, and still very quickly reach the 98
connection limit on postgres ... ALL from just harvesting mosaics
sequentially (not entire directory at once) in these 2 stores. So, the 98
connections I showed are all created from harvesting in these 2 stores.
My current datastore.properties for both stores is:
SPI=org.geotools.data.postgis.PostgisNGDataStoreFactory
host=localhost
port=5432
database=******
schema=mosaics
user=******
passwd=******
max\ connections=5
Loose\ bbox=true
Estimated\ extends=false
validate\ connections=true
Connection\ timeout=10
preparedStatements=true
So, does this not seem like connections are being leaked somewhere? Mosaic
harvesting not using the pool?
I wanted to break out of my script and make sure there wasn't anything
wrong with my workflow. So, I created the simplest mosaic I could with the
same datastore.properties file posted above. It consisted of 2 temperature
files from the geosolutions multidimensional tutorial in a directory called
"casefour'.
When I run the following command at the command line:
curl -v -u xxxxx:xxxxx -XPOST -H "Content-type: text/plain" -d
"file:/home/ubuntu/rest_harvesting/casefour"
http://localhost:8080/geoserver/rest/workspaces/geosolutions/coveragestores/casefour/external.imagemosaic
there is a 1 : 1 relationship every time I execute this command to a new
connection being opened to the database and remaining idle afterwards. If
I run that command 90 times, I end up with 90 connections open in the db.
max\ connections is set to 5 (default 10, right) ... so why should I see
more than 5 connections?
What else can I do to troubleshoot and help figure this out?
Thanks,
Mike
On Sat, Apr 11, 2015 at 2:20 AM, Andrea Aime <[email protected]>
wrote:
> On Fri, Apr 10, 2015 at 11:37 PM, Mike Grogan <[email protected]>
> wrote:
>
>> I see "select now()" being used as a validationQuery in lots of places in
>> the source. It appears, though, that database connections are not being
>> closed somewhere after harvesting via REST calls.
>>
>
> GeoServer uses connection pools, so it's normal that they are not closed,
> they are supposedly
> just given back to the pool.
>
> We need to understand if we are leaking them (not releasing them back to
> the pool), or if it's just
> that you setup the pool with too many connections, compared to what your
> dbms is willing
> to support.
>
> If it's the first, you'll see the code fail with an error stating it was
> not possible to get a connection
> from the pool... that does not seem your case.
>
> If it's the latter, we probably need to understand better your setup, how
> many mosaics, your
> datastore.properties, and so on.
> In general, if you have many dbms based mosaics, you will have a separate
> pool for each, which will
> quickly exhaust your connections server side, unless you setup a shared
> connection pool at the
> tomcat level, expose it via jndi, and refer to it that way.
> Examples here:
>
> http://geoserver.geo-solutions.it/multidim/en/imagemosaic/mosaic_datastore.html#using-a-jndi-connection-pool
>
> Cheers
> Andrea
>
> --
> ==
> GeoServer Professional Services from the experts! Visit
> http://goo.gl/NWWaa2 for more information.
> ==
>
> Ing. Andrea Aime
> @geowolf
> Technical Lead
>
> GeoSolutions S.A.S.
> Via Poggio alle Viti 1187
> 55054 Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax: +39 0584 1660272
> mob: +39 339 8844549
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>
> Le informazioni contenute in questo messaggio di posta elettronica e/o
> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
> darcene notizia via e-mail e di procedere alla distruzione del messaggio
> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
> principi dettati dal D.Lgs. 196/2003.
>
>
>
> The information in this message and/or attachments, is intended solely for
> the attention and use of the named addressee(s) and may be confidential or
> proprietary in nature or covered by the provisions of privacy act
> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
> Code).Any use not in accord with its purpose, any disclosure, reproduction,
> copying, distribution, or either dissemination, either whole or partial, is
> strictly forbidden except previous formal approval of the named
> addressee(s). If you are not the intended recipient, please contact
> immediately the sender by telephone, fax or e-mail and delete the
> information in this message that has been received in error. The sender
> does not give any warranty or accept liability as the content, accuracy or
> completeness of sent messages and accepts no responsibility for changes
> made after they were sent or for other risks which arise as a result of
> e-mail transmission, viruses, etc.
>
> -------------------------------------------------------
>
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Geoserver-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
