Hi Andrea,
I’ve never used it with anything like those numbers of layers/stores/etc. But
a few thoughts triggered from your post:
> One possible approach is to create "smart" UI components, that would show
a DropDown if the list is short (less than 1000 items?) but would switch
to a textbox with autocomplete otherwise, which would start proposing
autocomplete
values as the user types into it.
How about just add an “autocomplete” which is available for everyone to use, as
well as the drop-down. And then disable the drop-down when there are > 1000
items.
As a bonus, the autocomplete could probably also be used for things like
layer-searching.
>if you have 100k layers the first
request will try to build a WFS schema containing all the schemas of all
the vector layers.
And I thought it was slow with just a couple of hundred WFS features! How about
caching individual WFS schemas to disk and rebuilding only ones that have been
changed? Seems pretty obvious so I guess there’s a reason that won’t work. ☺
HT(try and)H.
Cheers,
Jonathan
From: Andrea Aime [mailto:[email protected]]
Sent: Monday, November 10, 2014 3:30 PM
To: Geoserver-devel
Subject: [Geoserver-devel] Improving scalability of UI (and other subsystems)
against very large catalogs
Hi,
we are looking at what makes GeoServer hard to use (or simply impossible to use)
when one has a very large amount of layers/styles/stores/workspaces configured,
and eventually stored in jdbcconfig.
Point in case, to give you an idea: 450 workspaces, 1000 stores, 100k+ layers
So the first thing that shows up is UI bits, the table pages, but the layer one,
are very slow: this is because Gabriel migrated the layer one so that filtering,
sorting and paging are done inside the catalog, but all others are not.
Migrating these is not too difficult, and seems to be rather mechanical, with
no visible consequences for the user (or else, one, the UI suddenly becomes
worth using ;-) )
Then there is a category of issues that are instead mixing togherer a full
listing of the catalog, coupled with the "wrong" UI: we dropdowns
or "right/left" choosers such as the style and security ones that end up
listing the name of all stores/layers/styles and whatnot.
In this case it's the UI element itself that is not really scaling up, it was
meant
for small lists.
For these cases I'm not totally sure what to do.
One possible approach is to create "smart" UI components, that would show
a DropDown if the list is short (less than 1000 items?) but would switch
to a textbox with autocomplete otherwise, which would start proposing
autocomplete
values as the user types into it.
Or it could be a read only box, with link and a dialog to choose from, and have
a table in the dialog.
These at least are the approaches that are relatively easy to implement.
A combo box looking like a drop down, but editable and with the ability to
shrink the contents of the drop down as one types would probably be best,
but I'm not sure about how that would be done in Wicket.
Finally, there is some very nasty issue in WFS, if you have 100k layers the
first
request will try to build a WFS schema containing all the schemas of all
the vector layers.
Do we have any indication on how to proceed, in order to make requests
work only with the feature types needed, instead of having to build a
Godzilla schema and having to drop it every time anything in the config
changes (feature type config, new ft, ft removed)
There are more issues, like CascadeDeleteVisitor doing linear scan,
or capabilities document generation grabbing a list instead of scrolling over
an iterator, that needs to be address, as well, but let's say these are not
as visible as the issues above, you need some load and very large
caps documents to get into troubles with these, fortunately in our case
the layers are split among many workspaces, which make things
quite a bit more manageable.
Anyways... feedback, suggestions, very important bits that we migth
have missed?
Cheers
Andrea
--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
AVVERTENZE AI SENSI DEL D.Lgs. 196/2003
Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo
è consentito esclusivamente al destinatario del messaggio, per le finalità
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro
sistema. Conservare il messaggio stesso, divulgarlo anche in parte,
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse,
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for the
attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act (Legislative
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in
accord with its purpose, any disclosure, reproduction, copying, distribution,
or either dissemination, either whole or partial, is strictly forbidden except
previous formal approval of the named addressee(s). If you are not the intended
recipient, please contact immediately the sender by telephone, fax or e-mail
and delete the information in this message that has been received in error. The
sender does not give any warranty or accept liability as the content, accuracy
or completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of e-mail
transmission, viruses, etc.
-------------------------------------------------------
This message has been scanned for viruses by
MailControl<http://www.mailcontrol.com/>, a service from BlackSpider Technology
Click
here<https://www.mailcontrol.com/sr/UnvjdVzoCGHGX2PQPOmvUoRq5N+0kMbNBZMXldDyKheMamL0OPLc1AAM3OsLei9chu5YqLyvUFSJRcENOpaP7A==>
to report this email as spam.
________________________________
HR Wallingford and its subsidiaries uses faxes and emails for confidential and
legally privileged business communications. They do not of themselves create
legal commitments. Disclosure to parties other than addressees requires our
specific consent. We are not liable for unauthorised disclosures nor reliance
upon them.
If you have received this message in error please advise us immediately and
destroy all copies of it.
HR Wallingford Limited
Howbery Park, Wallingford, Oxfordshire, OX10 8BA, United Kingdom
Registered in England No. 02562099
________________________________
------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
