Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces( > 3h )

2016-12-19 Thread shshin 
Dear Andrea, 

Thanks for this. 

My colleagues have tested nightly build version with 3,000 layers from just one 
data store and got the results like this. It took around 3 minutes and 40 
seconds as opposed to 20 minutes with old GeoServer 2.10. It seems there’s much 
improvement. 

However my team also found that it took only 20 seconds in loading 9,000 layers 
with GeoServer 2.8.x without GWC. As you pointed out, we also found that 
there’s something wrong in GWC, so my team recompiled GeoServer 2.8.x without 
GWC to increase loading speed. And got the drastically reduced loading time to 
only 20 seconds. 

One interesting result is that when we removed GWC from nightly build version, 
the loading time was almost the same as nightly version’s with GWC. 

In short, the loading speed has been improved much with nightly version, 
however still there’s something to be improved. We’ll also dig into it. 

Kind regards, 
신상희 드림 
---
Shin, Sanghee
Gaia3D, Inc. - The GeoSpatial Company
http://www.gaia3d.com 

보낸 사람: Andrea Aime
보낸 날짜: 2016년 12월 19일 월요일 오전 4:59
받는 사람: Jaroslav Urik
참조: geoserver-users@lists.sourceforge.net
제목: Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces( > 3h 
)

On Sun, Dec 18, 2016 at 6:57 PM, Jaroslav Urik  wrote:
Hi Andrea,
 
sorry for this late answer, I was AFK on vacation for last 14 days.
I would like to thank you for all this work you have done!
I am really looking forward to testing this

One of the two pull requests is still pending review, I guess nobody is worried
and I'll move on with merging it. This one is about having many stores (probably
more or your case).

The other one was about loading with many layers but few stores instead,
and got merged already, you can try it out using a nightly build, or just wait
for the 2.10.1 release (should be a few days).

Generally speaking best to try out early on a nightly build, when the release 
is out,
it's out, and then next one from the same series is going to be in two months

Cheers
Andrea

-- 
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime 
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i 
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo 
è consentito esclusivamente al destinatario del messaggio, per le finalità 
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne 
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di 
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro 
sistema. Conservare il messaggio stesso, divulgarlo anche in parte, 
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, 
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
 
The information in this message and/or attachments, is intended solely for the 
attention and use of the named addressee(s) and may be confidential or 
proprietary in nature or covered by the provisions of privacy act (Legislative 
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in 
accord with its purpose, any disclosure, reproduction, copying, distribution, 
or either dissemination, either whole or partial, is strictly forbidden except 
previous formal approval of the named addressee(s). If you are not the intended 
recipient, please contact immediately the sender by telephone, fax or e-mail 
and delete the information in this message that has been received in error. The 
sender does not give any warranty or accept liability as the content, accuracy 
or completeness of sent messages and accepts no responsibility  for changes 
made after they were sent or for other risks which arise as a result of e-mail 
transmission, viruses, etc.

---

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )

2016-12-18 Thread Andrea Aime 
On Sun, Dec 18, 2016 at 6:57 PM, Jaroslav Urik  wrote:

> Hi Andrea,
>
> sorry for this late answer, I was AFK on vacation for last 14 days.
> I would like to thank you for all this work you have done!
> I am really looking forward to testing this
>

One of the two pull requests is still pending review, I guess nobody is
worried
and I'll move on with merging it. This one is about having many stores
(probably
more or your case).

The other one was about loading with many layers but few stores instead,
and got merged already, you can try it out using a nightly build, or just
wait
for the 2.10.1 release (should be a few days).

Generally speaking best to try out early on a nightly build, when the
release is out,
it's out, and then next one from the same series is going to be in two
months

Cheers
Andrea

-- 
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.



The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility  for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

---
--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )

2016-12-18 Thread Jaroslav Urik 
Hi Andrea,

sorry for this late answer, I was AFK on vacation for last 14 days.
I would like to thank you for all this work you have done!
I am really looking forward to testing this

Thank you very much and have a nice Christmas =)

Kind regards,

Jaroslav



On Sat, Dec 10, 2016 at 11:40 AM Andrea Aime 
wrote:

> Hi,
> for those interested, I've prepared a couple of pull requests to deal with
> these issues. Not sure
> when they are going to land on 2.10 and 2.9, right now they are pending
> review:
> https://github.com/geoserver/geoserver/pull/1999
> https://github.com/geoserver/geoserver/pull/2008
>
> Cheers
> Andrea
>
>
> On Tue, Dec 6, 2016 at 12:26 PM, Andrea Aime  > wrote:
>
> I'm afraid this won't help as the code I've discovered is in the GWC
> integration and independent of
> how the storage is setup.
> The only way to get rid of it without changing the code would be to remove
> all gwc jars from GeoServer (which is btw completely feasible, if one does
> not need caching/WMTS)
>
> Cheers
> Andrea
>
> On Mon, Dec 5, 2016 at 8:46 PM, kweis...@harris.com 
> wrote:
>
> I haven’t tested this in a while, but we experienced this problem in the
> past and discovered that using the JDBC-Config plugin (from community
> modules) resolved the issue.  There is an initial hit during the first run
> with JDBC-Config enabled as it reads every resource file, validates it, and
> then inserts into the database, but then on subsequent startups the system
> is ready almost instantaneously.
>
> Kevin M. Weiss
> Software Engineer
> CRITICAL NETWORKS / HARRIS CORPORATION
> harris.com  / kevin.we...@harris.com
>
>
> From: Jaroslav Urik [mailto:jarda.u...@gmail.com]
> Sent: Thursday, December 01, 2016 8:30 AM
> To: Julian Hollingbery ;
> geoserver-users@lists.sourceforge.net
> Subject: Re: [Geoserver-users] GeoServer Slow to Start with Lots of
> Namespaces ( > 3h )
>
> Yes, that might solve it, but I do not know how to achieve this.. Also I
> hope that geonetwork just check if file exist and does not read the whole
> file..
>
> On Thu, Dec 1, 2016 at 3:09 PM Julian Hollingbery 
> wrote:
> … or possibly running these checks in parallel?
>
> Fra: Jaroslav Urik [mailto:jarda.u...@gmail.com]
> Sendt: 1. december 2016 14:36
> Til: geoserver-users@lists.sourceforge.net
> Emne: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces (
> > 3h )
>
> Hi,
> I have the same problem as described in [1] -- I am running GeoServer
> under Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes
> very long time to start - about 3-4 hours.
> The problem seems to be in that GeoServer is checking all of those
> resources one by one each time it starts..
> My question is: Is there any way to disable these checks? Or do them only
> one a month or so?
> Or is there any other solution?
> It is quite annoying when server needs to be restarted and people are
> waiting for it..
>
> Thanks in advance
>
> Jaroslav Urik
>
>
> [1]
> http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-Start-with-Lots-of-Namespaces-td5068101.html
>
> --
> ___
> Geoserver-users mailing list
> Geoserver-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>
>
>
>
> --
> ==
> GeoServer Professional Services from the experts! Visit
> http://goo.gl/it488V for more information.
> ==
>
> Ing. Andrea Aime
> @geowolf
> Technical Lead
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
> phone: +39 0584 962313 <+39%200584%20962313>
> fax: +39 0584 1660272 <+39%200584%20166%200272>
> mob: +39  339 8844549 <+39%20339%20884%204549>
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>
> Le informazioni contenute in questo messaggio di posta elettronica e/o
> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
> darcene notizia via e-mail e di procedere alla distruzione del messaggio
> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
> principi dettati dal D.Lgs. 196/2003.
>
>
>
> The information in this message and/or attachments, is intended solely for
> the attention and use of the named addressee(s) and may be confidential or
> proprietary in nature or covered by the provisions of privacy act
> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
> Code).Any use not in

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )

2016-12-10 Thread Andrea Aime 
Hi,
for those interested, I've prepared a couple of pull requests to deal with
these issues. Not sure
when they are going to land on 2.10 and 2.9, right now they are pending
review:
https://github.com/geoserver/geoserver/pull/1999
https://github.com/geoserver/geoserver/pull/2008

Cheers
Andrea


On Tue, Dec 6, 2016 at 12:26 PM, Andrea Aime 
wrote:

> I'm afraid this won't help as the code I've discovered is in the GWC
> integration and independent of
> how the storage is setup.
> The only way to get rid of it without changing the code would be to remove
> all gwc jars from GeoServer (which is btw completely feasible, if one does
> not need caching/WMTS)
>
> Cheers
> Andrea
>
> On Mon, Dec 5, 2016 at 8:46 PM, kweis...@harris.com 
> wrote:
>
>> I haven’t tested this in a while, but we experienced this problem in the
>> past and discovered that using the JDBC-Config plugin (from community
>> modules) resolved the issue.  There is an initial hit during the first run
>> with JDBC-Config enabled as it reads every resource file, validates it, and
>> then inserts into the database, but then on subsequent startups the system
>> is ready almost instantaneously.
>>
>> Kevin M. Weiss
>> Software Engineer
>> CRITICAL NETWORKS / HARRIS CORPORATION
>> harris.com  / kevin.we...@harris.com
>>
>>
>> From: Jaroslav Urik [mailto:jarda.u...@gmail.com]
>> Sent: Thursday, December 01, 2016 8:30 AM
>> To: Julian Hollingbery ;
>> geoserver-users@lists.sourceforge.net
>> Subject: Re: [Geoserver-users] GeoServer Slow to Start with Lots of
>> Namespaces ( > 3h )
>>
>> Yes, that might solve it, but I do not know how to achieve this.. Also I
>> hope that geonetwork just check if file exist and does not read the whole
>> file..
>>
>> On Thu, Dec 1, 2016 at 3:09 PM Julian Hollingbery 
>> wrote:
>> … or possibly running these checks in parallel?
>>
>> Fra: Jaroslav Urik [mailto:jarda.u...@gmail.com]
>> Sendt: 1. december 2016 14:36
>> Til: geoserver-users@lists.sourceforge.net
>> Emne: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces (
>> > 3h )
>>
>> Hi,
>> I have the same problem as described in [1] -- I am running GeoServer
>> under Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes
>> very long time to start - about 3-4 hours.
>> The problem seems to be in that GeoServer is checking all of those
>> resources one by one each time it starts..
>> My question is: Is there any way to disable these checks? Or do them only
>> one a month or so?
>> Or is there any other solution?
>> It is quite annoying when server needs to be restarted and people are
>> waiting for it..
>>
>> Thanks in advance
>>
>> Jaroslav Urik
>>
>>
>> [1] http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-Start-
>> with-Lots-of-Namespaces-td5068101.html
>> 
>> --
>> ___
>> Geoserver-users mailing list
>> Geoserver-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>>
>
>
>
> --
> ==
> GeoServer Professional Services from the experts! Visit
> http://goo.gl/it488V for more information.
> ==
>
> Ing. Andrea Aime
> @geowolf
> Technical Lead
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
> phone: +39 0584 962313 <+39%200584%20962313>
> fax: +39 0584 1660272 <+39%200584%20166%200272>
> mob: +39  339 8844549 <+39%20339%20884%204549>
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>
> Le informazioni contenute in questo messaggio di posta elettronica e/o
> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
> darcene notizia via e-mail e di procedere alla distruzione del messaggio
> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
> principi dettati dal D.Lgs. 196/2003.
>
>
>
> The information in this message and/or attachments, is intended solely for
> the attention and use of the named addressee(s) and may be confidential or
> proprietary in nature or covered by the provisions of privacy act
> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
> Code).Any use not in accord with its purpose, any disclosure, reproduction,
> copying, distribution, or either dissemination, either whole or partial, is
> strictly forbidden except previous formal approval of the named
> addressee(s). If you are not the intended recipient, please contact
> immediately the sender by telephone,

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )

2016-12-06 Thread Andrea Aime 
I'm afraid this won't help as the code I've discovered is in the GWC
integration and independent of
how the storage is setup.
The only way to get rid of it without changing the code would be to remove
all gwc jars from GeoServer (which is btw completely feasible, if one does
not need caching/WMTS)

Cheers
Andrea

On Mon, Dec 5, 2016 at 8:46 PM, kweis...@harris.com 
wrote:

> I haven’t tested this in a while, but we experienced this problem in the
> past and discovered that using the JDBC-Config plugin (from community
> modules) resolved the issue.  There is an initial hit during the first run
> with JDBC-Config enabled as it reads every resource file, validates it, and
> then inserts into the database, but then on subsequent startups the system
> is ready almost instantaneously.
>
> Kevin M. Weiss
> Software Engineer
> CRITICAL NETWORKS / HARRIS CORPORATION
> harris.com  / kevin.we...@harris.com
>
>
> From: Jaroslav Urik [mailto:jarda.u...@gmail.com]
> Sent: Thursday, December 01, 2016 8:30 AM
> To: Julian Hollingbery ; geoserver-users@lists.
> sourceforge.net
> Subject: Re: [Geoserver-users] GeoServer Slow to Start with Lots of
> Namespaces ( > 3h )
>
> Yes, that might solve it, but I do not know how to achieve this.. Also I
> hope that geonetwork just check if file exist and does not read the whole
> file..
>
> On Thu, Dec 1, 2016 at 3:09 PM Julian Hollingbery 
> wrote:
> … or possibly running these checks in parallel?
>
> Fra: Jaroslav Urik [mailto:jarda.u...@gmail.com]
> Sendt: 1. december 2016 14:36
> Til: geoserver-users@lists.sourceforge.net
> Emne: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces (
> > 3h )
>
> Hi,
> I have the same problem as described in [1] -- I am running GeoServer
> under Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes
> very long time to start - about 3-4 hours.
> The problem seems to be in that GeoServer is checking all of those
> resources one by one each time it starts..
> My question is: Is there any way to disable these checks? Or do them only
> one a month or so?
> Or is there any other solution?
> It is quite annoying when server needs to be restarted and people are
> waiting for it..
>
> Thanks in advance
>
> Jaroslav Urik
>
>
> [1] http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-
> Start-with-Lots-of-Namespaces-td5068101.html
> 
> --
> ___
> Geoserver-users mailing list
> Geoserver-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>



-- 
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.



The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility  for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

---
--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )

2016-12-05 Thread kweis...@harris.com 
I haven’t tested this in a while, but we experienced this problem in the past 
and discovered that using the JDBC-Config plugin (from community modules) 
resolved the issue.  There is an initial hit during the first run with 
JDBC-Config enabled as it reads every resource file, validates it, and then 
inserts into the database, but then on subsequent startups the system is ready 
almost instantaneously.  

Kevin M. Weiss
Software Engineer
CRITICAL NETWORKS / HARRIS CORPORATION
harris.com  / kevin.we...@harris.com


From: Jaroslav Urik [mailto:jarda.u...@gmail.com] 
Sent: Thursday, December 01, 2016 8:30 AM
To: Julian Hollingbery ; 
geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces 
( > 3h )

Yes, that might solve it, but I do not know how to achieve this.. Also I hope 
that geonetwork just check if file exist and does not read the whole file..

On Thu, Dec 1, 2016 at 3:09 PM Julian Hollingbery  wrote:
… or possibly running these checks in parallel?
 
Fra: Jaroslav Urik [mailto:jarda.u...@gmail.com] 
Sendt: 1. december 2016 14:36
Til: geoserver-users@lists.sourceforge.net
Emne: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )
 
Hi, 
I have the same problem as described in [1] -- I am running GeoServer under 
Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes very long 
time to start - about 3-4 hours. 
The problem seems to be in that GeoServer is checking all of those resources 
one by one each time it starts..
My question is: Is there any way to disable these checks? Or do them only one a 
month or so? 
Or is there any other solution?
It is quite annoying when server needs to be restarted and people are waiting 
for it..
 
Thanks in advance
 
Jaroslav Urik
 
 
[1] 
http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-Start-with-Lots-of-Namespaces-td5068101.html
--
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces( > 3h )

2016-12-04 Thread Andrea Aime 
Oook, think I've found it... yesterday I thought the startup was complete,
but I was wrong (was late, got tired).
In the trailing end of the startup GWC there is a check verifying that each
vector layer has as geometry
column (in other terms, that it's not geometryless).

This triggers the computation of the table structure for each and every
layer, taking ages and keeping
the database really busy for a long time.

Unfortunately I cannot talk with the original developer adding this code as
he left, but it seems to me
that a geometryless layer should not be allowed caching at creation time,
and verification at startup
time should be useless.

I was not involved in reviewing the original pull request, I'll check with
the ones that had a look at it:
https://github.com/geoserver/geoserver/pull/837

Cheers
Andrea


On Sat, Dec 3, 2016 at 9:55 PM, Andrea Aime 
wrote:

> Hi,
> I've tried to setup locally an artificial test case with lots of layers,
> checking if by any chance it's just that
> which is making things "bad".
> Local setup:
>
>- GeoServer development series (2.11.x)
>- Postgresql/Postgis
>- One store, one database
>- 1 layers, each one with its own style
>
>
> - Startup time: 21:45:20
> - All layers, store and styles loaded at 21:45:43
> - Startup complete at 21:45:59
> > total startup time, less than 30 seconds
>
> Now, by this I'm inclined to assume the issue is with having not a lot of
> layers, but a lot of stores,
> and the sanity check that GeoServer does when loading the store (there are
> none for the layers per se).
> The thing is, there is nothing new about these checks, they have been
> there for 10+ years.. it
> may be that what we do during this check has become more expensive
> recently.
>
> But it also points out clearly that a random setup with lots of layers
> does not incurs in the slow
> startup, and there is a need for more details on how to reproduce.
>
> Please everybody involved, share:
>
>- Version of GeoServer that starts fast, version that starts slow
>- How many workspaces
>- How many stores per workspace, and which type. Is there any type
>that feels slower than others?
>- If the store has the schema configured, or left blank
>- When configuring a new layer out of the store, how many tables are
>showing up?
>
>
> Cheers
> Andrea
>
>
> On Fri, Dec 2, 2016 at 2:53 AM,  wrote:
>
>> Hi Andrea,
>>
>>
>>
>> I filed a ticket since I think this symptom is very similar to the one I
>> posted a couple of days ago. I talked with Andy who experienced similar
>> issue but found no solution. Hope this get fixed soon.
>>
>>
>>
>> Warm regards,
>>
>> 신상희 드림
>>
>> [1] https://osgeo-org.atlassian.net/browse/GEOS-7884
>>
>> ---
>> Shin, Sanghee
>> Gaia3D, Inc. - The GeoSpatial Company
>> http://www.gaia3d.com
>>
>>
>>
>> *보낸 사람: *Andrea Aime 
>> *보낸 날짜: *2016년 12월 2일 금요일 오전 9:49
>> *받는 사람: *Jaroslav Urik 
>> *참조: *GeoServer Mailing List List 
>> *제목: *Re: [Geoserver-users] GeoServer Slow to Start with Lots of
>> Namespaces( > 3h )
>>
>>
>>
>> Hi,
>>
>> we don't know why it's happening but you're not the first to report it.
>>
>> I already asked in vain for someone to report a ticket with some details
>> at https://osgeo-org.atlassian.net/projects/GEOS/issues ?
>>
>> If there is no bug report, for sure nobody will look into it when the
>> monthly bug fix code sprint comes (if there is it's still no guarantee,
>>
>> but at least a chance).
>>
>>
>>
>> Cheers
>>
>> Andrea
>>
>>
>>
>> On Thu, Dec 1, 2016 at 2:36 PM, Jaroslav Urik 
>> wrote:
>>
>> Hi,
>>
>> I have the same problem as described in [1] -- I am running GeoServer
>> under Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes
>> very long time to start - about 3-4 hours.
>>
>> The problem seems to be in that GeoServer is checking all of those
>> resources one by one each time it starts..
>>
>> My question is: Is there any way to disable these checks? Or do them only
>> one a month or so?
>>
>> Or is there any other solution?
>>
>> It is quite annoying when server needs to be restarted and people are
>> waiting for it..
>>
>>
>>
>> Thanks in advance
>>
>>
>>
>> Jaroslav Urik
>>
>>
>>
>>
>>
>> [1] http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-Start-
>> with-Lots-of-Namespaces-td5068101.html
>>
>>
>> 
>> --
>>
>> ___
>> Geoserver-users mailing list
>> Geoserver-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>>
>>
>>
>>
>>
>> --
>>
>> ==
>>
>> GeoServer Professional Services from the experts! Visit
>>
>> http://goo.gl/it488V for more information.
>>
>> ==
>>
>>
>>
>> Ing. Andrea Aime
>>
>> @geowolf
>>
>> Technical Lead
>>
>>
>>
>>

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces( > 3h )

2016-12-03 Thread Andrea Aime 
Hi,
I've tried to setup locally an artificial test case with lots of layers,
checking if by any chance it's just that
which is making things "bad".
Local setup:

   - GeoServer development series (2.11.x)
   - Postgresql/Postgis
   - One store, one database
   - 1 layers, each one with its own style


- Startup time: 21:45:20
- All layers, store and styles loaded at 21:45:43
- Startup complete at 21:45:59
> total startup time, less than 30 seconds

Now, by this I'm inclined to assume the issue is with having not a lot of
layers, but a lot of stores,
and the sanity check that GeoServer does when loading the store (there are
none for the layers per se).
The thing is, there is nothing new about these checks, they have been there
for 10+ years.. it
may be that what we do during this check has become more expensive recently.

But it also points out clearly that a random setup with lots of layers does
not incurs in the slow
startup, and there is a need for more details on how to reproduce.

Please everybody involved, share:

   - Version of GeoServer that starts fast, version that starts slow
   - How many workspaces
   - How many stores per workspace, and which type. Is there any type that
   feels slower than others?
   - If the store has the schema configured, or left blank
   - When configuring a new layer out of the store, how many tables are
   showing up?


Cheers
Andrea


On Fri, Dec 2, 2016 at 2:53 AM,  wrote:

> Hi Andrea,
>
>
>
> I filed a ticket since I think this symptom is very similar to the one I
> posted a couple of days ago. I talked with Andy who experienced similar
> issue but found no solution. Hope this get fixed soon.
>
>
>
> Warm regards,
>
> 신상희 드림
>
> [1] https://osgeo-org.atlassian.net/browse/GEOS-7884
>
> ---
> Shin, Sanghee
> Gaia3D, Inc. - The GeoSpatial Company
> http://www.gaia3d.com
>
>
>
> *보낸 사람: *Andrea Aime 
> *보낸 날짜: *2016년 12월 2일 금요일 오전 9:49
> *받는 사람: *Jaroslav Urik 
> *참조: *GeoServer Mailing List List 
> *제목: *Re: [Geoserver-users] GeoServer Slow to Start with Lots of
> Namespaces( > 3h )
>
>
>
> Hi,
>
> we don't know why it's happening but you're not the first to report it.
>
> I already asked in vain for someone to report a ticket with some details
> at https://osgeo-org.atlassian.net/projects/GEOS/issues ?
>
> If there is no bug report, for sure nobody will look into it when the
> monthly bug fix code sprint comes (if there is it's still no guarantee,
>
> but at least a chance).
>
>
>
> Cheers
>
> Andrea
>
>
>
> On Thu, Dec 1, 2016 at 2:36 PM, Jaroslav Urik 
> wrote:
>
> Hi,
>
> I have the same problem as described in [1] -- I am running GeoServer
> under Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes
> very long time to start - about 3-4 hours.
>
> The problem seems to be in that GeoServer is checking all of those
> resources one by one each time it starts..
>
> My question is: Is there any way to disable these checks? Or do them only
> one a month or so?
>
> Or is there any other solution?
>
> It is quite annoying when server needs to be restarted and people are
> waiting for it..
>
>
>
> Thanks in advance
>
>
>
> Jaroslav Urik
>
>
>
>
>
> [1] http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-
> Start-with-Lots-of-Namespaces-td5068101.html
>
>
> 
> --
>
> ___
> Geoserver-users mailing list
> Geoserver-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>
>
>
>
>
> --
>
> ==
>
> GeoServer Professional Services from the experts! Visit
>
> http://goo.gl/it488V for more information.
>
> ==
>
>
>
> Ing. Andrea Aime
>
> @geowolf
>
> Technical Lead
>
>
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
>
> phone: +39 0584 962313 <+39%200584%20962313>
>
> fax: +39 0584 1660272 <+39%200584%20166%200272>
>
> mob: +39  339 8844549 <+39%20339%20884%204549>
>
>
>
> http://www.geo-solutions.it
>
> http://twitter.com/geosolutions_it
>
>
>
> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>
> Le informazioni contenute in questo messaggio di posta elettronica e/o
> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
> darcene notizia via e-mail e di procedere alla distruzione del messaggio
> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
> principi dettati dal D.Lgs. 196/2003.
>
>
>
> The information in this message and/or attachments, is

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces( > 3h )

2016-12-01 Thread shshin 
Hi Andrea, 

I filed a ticket since I think this symptom is very similar to the one I posted 
a couple of days ago. I talked with Andy who experienced similar issue but 
found no solution. Hope this get fixed soon. 

Warm regards, 
신상희 드림 

[1] https://osgeo-org.atlassian.net/browse/GEOS-7884 
---
Shin, Sanghee
Gaia3D, Inc. - The GeoSpatial Company
http://www.gaia3d.com 

보낸 사람: Andrea Aime
보낸 날짜: 2016년 12월 2일 금요일 오전 9:49
받는 사람: Jaroslav Urik
참조: GeoServer Mailing List List
제목: Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces( > 3h 
)

Hi,
we don't know why it's happening but you're not the first to report it.
I already asked in vain for someone to report a ticket with some details at 
https://osgeo-org.atlassian.net/projects/GEOS/issues ?
If there is no bug report, for sure nobody will look into it when the monthly 
bug fix code sprint comes (if there is it's still no guarantee,
but at least a chance).

Cheers
Andrea

On Thu, Dec 1, 2016 at 2:36 PM, Jaroslav Urik  wrote:
Hi, 
I have the same problem as described in [1] -- I am running GeoServer under 
Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes very long 
time to start - about 3-4 hours. 
The problem seems to be in that GeoServer is checking all of those resources 
one by one each time it starts..
My question is: Is there any way to disable these checks? Or do them only one a 
month or so? 
Or is there any other solution?
It is quite annoying when server needs to be restarted and people are waiting 
for it..

Thanks in advance

Jaroslav Urik


[1] 
http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-Start-with-Lots-of-Namespaces-td5068101.html

--

___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users




-- 
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime 
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i 
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo 
è consentito esclusivamente al destinatario del messaggio, per le finalità 
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne 
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di 
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro 
sistema. Conservare il messaggio stesso, divulgarlo anche in parte, 
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, 
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
 
The information in this message and/or attachments, is intended solely for the 
attention and use of the named addressee(s) and may be confidential or 
proprietary in nature or covered by the provisions of privacy act (Legislative 
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in 
accord with its purpose, any disclosure, reproduction, copying, distribution, 
or either dissemination, either whole or partial, is strictly forbidden except 
previous formal approval of the named addressee(s). If you are not the intended 
recipient, please contact immediately the sender by telephone, fax or e-mail 
and delete the information in this message that has been received in error. The 
sender does not give any warranty or accept liability as the content, accuracy 
or completeness of sent messages and accepts no responsibility  for changes 
made after they were sent or for other risks which arise as a result of e-mail 
transmission, viruses, etc.

---

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )

2016-12-01 Thread Andrea Aime 
On Thu, Dec 1, 2016 at 2:36 PM, Jaroslav Urik  wrote:

> Hi,
> I have the same problem as described in [1] -- I am running GeoServer
> under Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes
> very long time to start - about 3-4 hours.
> The problem seems to be in that GeoServer is checking all of those
> resources one by one each time it starts..
> My question is: Is there any way to disable these checks? Or do them only
> one a month or so?
>

Btw, forgot about this part. The checks cannot be disabled right now, they
are there to protect the capabilites
document generation (that used to fail if only one layer/store was
misconfigured).
Now we have an option to make the caps document generation "tolerant", if
one layer fails it will still go on skipping it,
but there is no setting to disable the startup checks yet.

We'd need to write some code to control that, a new settings in the global
server options. Please open a ticket
https://osgeo-org.atlassian.net/projects/GEOS/summary requesting that
functionality.

Still, 65k layers should not take that long, even with the checks how
many workspaces and stores do you have?

Cheers
Andrea

-- 
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.



The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility  for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

---
--
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )

2016-12-01 Thread Andrea Aime 
Hi,
we don't know why it's happening but you're not the first to report it.
I already asked in vain for someone to report a ticket with some details at
https://osgeo-org.atlassian.net/projects/GEOS/issues ?
If there is no bug report, for sure nobody will look into it when the
monthly bug fix code sprint comes (if there is it's still no guarantee,
but at least a chance).

Cheers
Andrea

On Thu, Dec 1, 2016 at 2:36 PM, Jaroslav Urik  wrote:

> Hi,
> I have the same problem as described in [1] -- I am running GeoServer
> under Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes
> very long time to start - about 3-4 hours.
> The problem seems to be in that GeoServer is checking all of those
> resources one by one each time it starts..
> My question is: Is there any way to disable these checks? Or do them only
> one a month or so?
> Or is there any other solution?
> It is quite annoying when server needs to be restarted and people are
> waiting for it..
>
> Thanks in advance
>
> Jaroslav Urik
>
>
> [1] http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-
> Start-with-Lots-of-Namespaces-td5068101.html
>
> 
> --
>
> ___
> Geoserver-users mailing list
> Geoserver-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>
>


-- 
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.



The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility  for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.

---
--
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )

2016-12-01 Thread Jaroslav Urik 
Yes, that might solve it, but I do not know how to achieve this.. Also I
hope that geonetwork just check if file exist and does not read the whole
file..

On Thu, Dec 1, 2016 at 3:09 PM Julian Hollingbery 
wrote:

> … or possibly running these checks in parallel?
>
>
>
> *Fra:* Jaroslav Urik [mailto:jarda.u...@gmail.com]
> *Sendt:* 1. december 2016 14:36
> *Til:* geoserver-users@lists.sourceforge.net
> *Emne:* [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces
> ( > 3h )
>
>
>
> Hi,
>
> I have the same problem as described in [1] -- I am running GeoServer
> under Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes
> very long time to start - about 3-4 hours.
>
> The problem seems to be in that GeoServer is checking all of those
> resources one by one each time it starts..
>
> My question is: Is there any way to disable these checks? Or do them only
> one a month or so?
>
> Or is there any other solution?
>
> It is quite annoying when server needs to be restarted and people are
> waiting for it..
>
>
>
> Thanks in advance
>
>
>
> Jaroslav Urik
>
>
>
>
>
> [1]
> http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-Start-with-Lots-of-Namespaces-td5068101.html
>
--
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )

2016-12-01 Thread Julian Hollingbery 
… or possibly running these checks in parallel?

Fra: Jaroslav Urik [mailto:jarda.u...@gmail.com]
Sendt: 1. december 2016 14:36
Til: geoserver-users@lists.sourceforge.net
Emne: [Geoserver-users] GeoServer Slow to Start with Lots of Namespaces ( > 3h )

Hi,
I have the same problem as described in [1] -- I am running GeoServer under 
Tomcat with a lot of georeferenced maps ( about 65 000 ) and it takes very long 
time to start - about 3-4 hours.
The problem seems to be in that GeoServer is checking all of those resources 
one by one each time it starts..
My question is: Is there any way to disable these checks? Or do them only one a 
month or so?
Or is there any other solution?
It is quite annoying when server needs to be restarted and people are waiting 
for it..

Thanks in advance

Jaroslav Urik


[1] 
http://osgeo-org.1560.x6.nabble.com/GeoServer-Slow-to-Start-with-Lots-of-Namespaces-td5068101.html
--
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users