Re: [openstack-dev] [oslo.db] Marker based paging

2014-11-06 Thread Heald, Mike
Unfortunately, ordering on anything but the timestamp
results in problems with usability and results consistency.

Example: Ordering on priority so I can see all my high priority tasks
first, then medium, then low.

Works ok, as long as the underlying data is geared so that the priorities
are ordered that way. But I have no way of skipping pages to find
them if they're not, and if the task that that we're using as the marker
changes priority as I page, I'm jumped to somewhere else in the list and I
have no way of knowing where I am in the list. I'd either see a load of
repeated data (with no way of skipping it)  or miss out on a lot of data
with no indication that this has happened.

So that's quite specific to Storyboard, but think of a situation where
you've ordered your instances in horizon by status so you can see
all that are spawning, running, deleting, etc. grouped together.
As you page, the marker's instance goes from spawning to running,
or running to deleting.

These aren't edge cases. The nature of the data we deal with forces
us to handle results that are going to change beneath us.

Now, I'm in no way suggesting that offset/limit would be a better solution!
But we do need to document these use cases somewhere and make sure
our paging solution can fulfil them, because Storyboard cannot provide the
user experience it needs to with the current pagination method in oslo.db.

Cheers,
Mike


From: Jay Pipes [jaypi...@gmail.com]
Sent: 04 November 2014 07:07
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [oslo.db] Marker based paging

On 11/04/2014 01:08 AM, Heald, Mike wrote:
 Thanks for that, Steven :)

 So just to clarify, results are ordered by the relevant timestamps to
 ensure consistent order and so that new records would never show on
 previous pages and be missed, and we're limited to just a next
 page navigation, and we cannot order the entire result set on any
 column but the timestamps, as this would break the paging because we
 can't do the comparisons we need to if the results aren't in that
 order. Have I got that correct?

No, that's not correct. There's nothing limiting one from ordering on
other columns than timestamp. We always ensure that there is a secondary
order on a column with unique values (like the primary key), in order to
ensure that pages of results are strictly ordered even when the sort
field is non-unique (like timestamp).

We're limited to next-previous pagination by choice because of the
scalability and performance limitations of a limit-offset pagination
strategy.

Best,
jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo.db] Marker based paging

2014-11-03 Thread Heald, Mike
Thanks for that, Steven :)

So just to clarify, results are ordered by the relevant timestamps to ensure 
consistent order and so that new records would never show on previous pages 
and be missed, and we're limited to just a next page navigation, and we 
cannot order the entire result set on any column but the timestamps, as this 
would break the paging because we can't do the comparisons we need to if the 
results aren't in that order. Have I got that correct?

Thanks,
Mike



From: Steven Kaufer [kau...@us.ibm.com]
Sent: 03 November 2014 22:39
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [oslo.db] Marker based paging


Here are a few ML threads on this topic:

http://lists.openstack.org/pipermail/openstack-dev/2014-March/030322.html
http://www.gossamer-threads.com/lists/openstack/dev/2777
http://lists.openstack.org/pipermail/openstack-dev/2013-November/018861.html

Thanks,

Steven Kaufer

Roman Podoliaka rpodoly...@mirantis.com wrote on 11/03/2014 10:35:21 AM:

 From: Roman Podoliaka rpodoly...@mirantis.com
 To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 Date: 11/03/2014 10:43 AM
 Subject: Re: [openstack-dev] [oslo.db] Marker based paging

 Hi Mike,

 I think that code was taken from Nova (or maybe some other project) as
 is and we haven't touched it since then.

 Please speak up - we want to know about all possible problems with
 current approach.

 Thanks,
 Roman

 On Fri, Oct 31, 2014 at 2:58 PM, Heald, Mike mike.he...@hp.com wrote:
  Hi all,
 
  I'm implementing paging on storyboard, and I wanted to ask why we
 decided to use marker based paging. I have some opinions on this,
 but I want to keep my mouth shut until I find out what problem it
 was solving :)
 
  Thanks,
  Mike
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [oslo.db] Marker based paging

2014-10-31 Thread Heald, Mike
Hi all,

I'm implementing paging on storyboard, and I wanted to ask why we decided to 
use marker based paging. I have some opinions on this, but I want to keep my 
mouth shut until I find out what problem it was solving :)

Thanks,
Mike


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev