Hi Andrea ! I'm starting with some tests (increasing the amount of layers,
and checking types, kind of sources, etc).
We have 1 workspace, with about 350k layers, and 111k stores.
- workspaces could help to define some improvement ? (instead of 1, to
have more). From our perspective, it's the same "environment", and no need
for that, but.. probably you know better that us what it could imply (only
considering the "catalog init process")
- when you mention 42000 asociated tile layers, you are referring to
GWC, right ? (so for each layer, you have an associated tile layer)
- as I understand, the catalog process will read each of all xml files
in data_dir (coveragestore.xml, layer.xml, featuretype.xml files, etc),
trying to check that layers are ok, etc. right ?...in the case of 111
stores, we are talking about tons of files, when you said : "...both offer
clustering of the catalog"... how do you imagine the JMS doing that
?...different servers, offering a part of the complete catalog ?...
Some of the questions are based on these comments from the url you shared
- “Large”: 1001 workspaces, 11000 stores (a mix of shapefiles, postgis,
directory of shapefile, single tiff, arcgrid, mosaics), 42000 layers and
42000 associated tile layers
Regards
Jose
2018-06-25 7:34 GMT-03:00 Andrea Aime <[email protected]>:
> Hi Josè,
> 3 hours for 300.000 layers seems too much, I was able to start a catalog
> with 50k layers in 3 minutes from "cold" storage (not even a file in cache)
> since GeoServe 2.11, http://blog.geoserver.org/2017/03/23/geoserver-2-11-
> 0-released/ ,
> so 300k should take around 20 minutes (the data structures involved are
> scalable)... but that's still way too much.
>
> So you should consider using jdbcconfig instead, and remove JMS, but mind,
> once you use jdbcconfig, you're stuck in it, there is no ready to use way
> to extract
> the configuration from the database once it's in there, and the module is
> unsupported, so I would advise you to also get commercial support to go
> along with it:
> http://geoserver.org/support/
>
> Best regards
> Andrea
>
>
> On Mon, Jun 25, 2018 at 12:20 PM, José Macchi <[email protected]>
> wrote:
>
>> Hi Andrea ! thanks for the quick response.
>>
>> So, what about having +300k layers in Geoserver and have to spend +3
>> hours to initialize it ?. Currently our system is using JMS and that is ok,
>> because that allow us to handle several Geoservers in order to response
>> users requests, but if we need to restart the server it takes a really long
>> time to reload the catalog (and that was the reason behind installing
>> JDBCStore, allowing the startup to be working with a database instead of
>> reading +300 xml files from a FS, that clearly it will take much more time).
>>
>> Is there some way in the JMS plugin to clusterize the catalog into small
>> pieces and allow each instance to handle parts of it (so startup it will
>> move faster) ???
>>
>> Thanks !
>>
>> 2018-06-25 4:50 GMT-03:00 Andrea Aime <[email protected]>:
>>
>>> Hi,
>>> the two community modules both offer clustering of the catalog, but in
>>> two different ways, so they are not meant
>>> to be installed together
>>>
>>> Cheers
>>> Andrea
>>>
>>>
>>> On Mon, Jun 25, 2018 at 5:09 AM, José Macchi <[email protected]>
>>> wrote:
>>>
>>>> Hello, does anyone installed and used succesfully both plugins in a
>>>> geoserver instance ?
>>>>
>>>> It seems that there are conflicts based on some
>>>> "geoserverLoader-related definition" when you try to have both installed.
>>>> (no problems when those plugins are installed alone)
>>>>
>>>> Regards
>>>>
>>>>
>>>> --
>>>> José Macchi
>>>>
>>>> Geo DevOps
>>>> Frontec S.A.
>>>>
>>>> ------------------------------------------------------------
>>>> ------------------
>>>> Check out the vibrant tech community on one of the world's most
>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>> _______________________________________________
>>>> Geoserver-users mailing list
>>>>
>>>> Please make sure you read the following two resources before posting to
>>>> this list:
>>>> - Earning your support instead of buying it, but Ian Turton:
>>>> http://www.ianturton.com/talks/foss4g.html#/
>>>> - The GeoServer user list posting guidelines:
>>>> http://geoserver.org/comm/userlist-guidelines.html
>>>>
>>>> If you want to request a feature or an improvement, also see this:
>>>> https://github.com/geoserver/geoserver/wiki/Successfully-req
>>>> uesting-and-integrating-new-features-and-improvements-in-GeoServer
>>>>
>>>>
>>>> [email protected]
>>>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Regards, Andrea Aime == GeoServer Professional Services from the
>>> experts! Visit http://goo.gl/it488V for more information. == Ing.
>>> Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito
>>> 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob:
>>> +39 339 8844549 http://www.geo-solutions.it
>>> http://twitter.com/geosolutions_it
>>> -------------------------------------------------------
>>> *Con riferimento alla normativa sul trattamento dei dati personali (Reg.
>>> UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
>>> precisa che ogni circostanza inerente alla presente email (il suo
>>> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
>>> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
>>> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
>>> operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
>>> This email is intended only for the person or entity to which it is
>>> addressed and may contain information that is privileged, confidential or
>>> otherwise protected from disclosure. We remind that - as provided by
>>> European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
>>> e-mail or the information herein by anyone other than the intended
>>> recipient is prohibited. If you have received this email by mistake, please
>>> notify us immediately by telephone or e-mail.*
>>>
>>
>>
>>
>> --
>> José Macchi
>>
>> Geo DevOps
>> Frontec S.A.
>>
>
>
>
> --
>
> Regards, Andrea Aime == GeoServer Professional Services from the experts!
> Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime
> @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054
> Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339
> 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it
> ------------------------------------------------------- *Con riferimento
> alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 -
> Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni
> circostanza inerente alla presente email (il suo contenuto, gli eventuali
> allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i
> destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per
> errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le
> sarei comunque grato se potesse darmene notizia. This email is intended
> only for the person or entity to which it is addressed and may contain
> information that is privileged, confidential or otherwise protected from
> disclosure. We remind that - as provided by European Regulation 2016/679
> “GDPR” - copying, dissemination or use of this e-mail or the information
> herein by anyone other than the intended recipient is prohibited. If you
> have received this email by mistake, please notify us immediately by
> telephone or e-mail.*
>
--
José Macchi
Geo DevOps
Frontec S.A.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-users mailing list
Please make sure you read the following two resources before posting to this
list:
- Earning your support instead of buying it, but Ian Turton:
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines:
http://geoserver.org/comm/userlist-guidelines.html
If you want to request a feature or an improvement, also see this:
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-users
