On 08/13/2013 01:51 PM, Miller, Mark M (EB SW Cloud - R&D - Corvallis)
wrote:
I have been following this exchange of ideas on how to solve/implement
pagination. I would ask you to keep in mind that a solution needs to take into
account a split LDAP/SQL backend (you are not always dealing with a single
Keystone SQL database). Having a split backend means that the query information
is divided between both backends and that you may not have as much flexibility
with the LDAP backend
Yes, absolutely understood and a good point.
In the case of engines that don't support filtering, ordering, or other
DB-like operations, then a pagination implementation in the controller
would have to be provided. Not efficient, but better than nothing.
-jay
Mark.
-----Original Message-----
From: Jay Pipes [mailto:[email protected]]
Sent: Tuesday, August 13, 2013 10:10 AM
To: [email protected]
Subject: Re: [openstack-dev] [keystone] Pagination
On 08/13/2013 12:55 PM, Lyle, David (Cloud Services) wrote:
The marker/limit pagination scheme is inferior.
A bold statement that flies in the face of experience and the work already done
in all the other projects.
>The use of page/page_size allows access to arbitrary pages, whereas
limit/marker only allows forward progress.
I don't see this as a particularly compelling use case considering the
performance manifestations of using LIMIT OFFSET pagination.
>In Horizon's use case, with page/page_size we can provide the user access to
any page they have already visited, rather than just the previous page (using
prev/next links returned in the response).
I don't see this as a particularly useful thing, but in any case, you could
still do this by keeping the markers for previous pages on the client (Horizon)
side.
The point of marker/limit is to eliminate poor performance of LIMIT OFFSET
queries and to force proper index usage in the listing queries.
You can see the original discussion about this from more than two years and
even see where I was originally arguing for a LIMIT OFFSET strategy and was
brought around to the current limit/marker strategy by the responses of Justin
Santa Barbara and Greg Holt:
https://lists.launchpad.net/openstack/msg02548.html
Best,
-jay
-David
On 08/13/2013 10:29 AM, Pipes, Jay wrote:
On 08/13/2013 03:05 AM, Yee, Guang wrote:
Passing the query parameters, whatever they are, into the driver if
the given driver supports pagination and allowing the driver to
override the manager default pagination functionality seem reasonable to me.
Please do use the standards that are supported in other OpenStack services
already: limit, marker, sort_key and sort_dir.
Pagination is meaningless without a sort key and direction, so picking a sensible
default for user/project records is good. I'd go with either created_at (what
Glance/Nova/Cinder use..) or with the user/project >UUID.
The Glance DB API pagination is well-documented and clean [1]. I highly
recommend it as a starting point.
Nova uses the same marker/limit/sort_key/sort_dir options for queries
that it allows pagination on. An example is the
instance_get_all_by_filters() call [2].
Cinder uses the same marker/limit/sort_key/sort_dir options for query
pagination as well. [3]
Finally, I'd consider supporting the standard change-since parameter for listing operations.
Both Nova [4] and Glance [5] support the parameter, which is useful for tools that poll the
APIs for "new" >events/records.
In short, go with what is already a standard in the other projects...
Best,
-jay
[1]
https://github.com/openstack/glance/blob/master/glance/db/sqlalchemy/
api.py#L429
[2]
https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.
py#L1709
[3]
https://github.com/openstack/cinder/blob/master/cinder/common/sqlalch
emyutils.py#L33
[4]
https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.
py#L1766
[5]
https://github.com/openstack/glance/blob/master/glance/db/sqlalchemy/
api.py#L618
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
