On 8/28/2012 11:23 AM, Petr Vobornik wrote:
Your possible solution does not address how many results are fetched
(unless I misunderstood).
>
If paging is enabled it doesn't, but it expects, that admin will disable
it for larger setups.For smaller setups it isn't of much an issue. If
paging is disabled, the limit is server 'search size limit' or
--sizelimit option supplied by Web UI.
I'm not sure how per-user preferences are handled in browsers, but don't
forget we now have session support in the server. Server session data is
available for use.
I was thinking about using browser local storage (basically key-value
DB). It has a benefit over session, that it survives a browser restart
but it should contain only non-sensitive data (other users may see it).
You mean browser cookie?
If you don't want to fetch every record to implement paging smartly in
conjunction with it's performance issues why not do the query on the
server, cache the results in the session, and have the RPC return the
total number of results plus a subset of the result. Another RPC could
retrieve the next/previous subset of results from the cached result in
the session.
I think most software do paging like this. I don't know the reasons for
not doing it that way first time. My solution was counting with that we
still don't want to do it. Endi do you know the reasons? Earlier
sessions didn't exists, but it is doable without them too.
Yes, at the time the sessions didn't exist and we needed a
quick/temporary solution. I agree ideally this should be done on the
server side, but how would the server maintain the paging information?
The server can keep the search result (either just the pkey list or the
entire entries) in memcached, but the result might be large and there
might be multiple users accessing multiple search pages, so the total
memory requirement could be large.
If we don't want the server to keep the search result, the server could
rerun the search and return only the requested page and discard most of
the results each time the user change page. This may affect server
performance.
We can also use Simple Paged Results, but if I understood correctly it
requires the httpd to maintain an open connection to the LDAP server for
each user and for each page. I'm not sure memcached can be used to move
the connection object among forked httpd processes. Also Simple Paged
Results can only go forward, so no Prev button unless somebody keeps the
results.
Another possibility is to use VLV, but it seems to require open
connection too and only works with a predefined filter.
Here's the page I wrote about this:
http://freeipa.org/page/IPAv3_Directory_Browsing
Currently we're using Option #2 but I'm not sure if we use SPR between
httpd and the LDAP server. But even with SPR the result is still bound
by some limits. It looks you have to connect as Directory Manager to get
the complete list of pkey list/entries. See the table in this page:
http://directory.fedoraproject.org/wiki/Simple_Paged_Results_Design
The current implementation (keeping the pkey list in the UI) is
conceptually similar to the front-end approach described in the above page.
I don't think there any need in JSON formatted data for pretty printing
with indentation. Is it an accident or oversight we're pretty printing
the JSON data in an RPC. For large data sets compression would be a win,
but most of our RPC communication is not large. Compression has an
overhead. With small data you'll use more cycles compressing and
uncompressing than you would sending verbose but small data blobs.
Compression should be an RPC option specified by either side of the
connection and the receiver should be prepared to conditionally
uncompress based on a flag value in the RPC. If you use server session
data to support paging you may not need to introduce compression since
you would only be passing one page of data at a time.
User specified page size (without limitation) is an absolute necessity.
I am frequently annoyed by web sites which either do not allow me to
specify page size or constrain it to ridiculous hard coded limits such
as 10, 20 or 30.
+1
Agreed, but if we implement the single continuous page I described in
the earlier email the page size won't be much of an issue anymore.
--
Endi S. Dewata
_______________________________________________
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
