On 11/19/2014 11:38 AM, Heald, Mike wrote:
Hello, fellow Storyboarders!
After some discussions with the folks working on oslo.db and going
over our requirements for paging with krotscheck, we came to the
conclusion that paging just based on a marker did not give us the
reliability or functionality that we need in Storyboard. I said I'd
outline the paging model I'd used before for companies whose
requirements were quite demanding (e.g. if we miss a record while
paging, we could potentially lose thousands in profits)
Pretty sure Storyboard paged results are not going to affect anyone's
profit margin.
Goals:
1) Paged list where records do not jump from page to page, even if
the underlying data changes, so that the user never misses records
that match search criteria, and never sees the records she is dealing
with change unexpectedly
Is this use case really something to be worried about? Honestly, I don't
think so.
> 2) Able to order the list by column
Nothing about marker/limit pagination prevents ordering by any column.
> 3) Able to jump to an arbitrary page
What about this last use case is useful? Why would I want to "jump to
the 7th page of results"? Just being able to do something doesn't
necessarily make it useful.
To fully satisfy (1) while having (2) and (3), we need to snapshot
the result set, and have a mechanism for notifying the user of any
new records that match the list criteria since they started
browsing.
This can be achieved by having a three column table, with a record
ID, the data used to order the results, and an ID for the result set.
This allows us to left join on the main data table (so that record
deletions do not affect paging, and the UI can show that a record was
deleted and not mess up paging). Ordering by the snapshotted data
means that if records change that data in the main data table, they
do not jump pages, and the UI can show that the data has changed.
This does require some resultset metadata too (e.g. column sorted
by)
Notifications of new records can be done with a web worker that
checks if there are records that do not appear in the search but do
match the search criteria. The user can be notified that new records
have come in, and opt to either view a list of these records, or
rerun the search to include them.
Downsides of this approach are: - It's relatively heavy. The search
table for each datatype will have lots of inserts and deletes, so
careful consideration of partitions, or use of temporary tables, or
other webscale stuff needs to happen - Housekeeping needs to be
thought about. Just viewing a list of stuff will result in a snapshot
so it can be manipulated correctly, so we need to think about if
there are situations when we *don't* want this to happen (for example
viewing a list of all stories might not warrant it, but viewing a
list of stories with a particular status, might), how to clean up
result set snapshots that are no longer in use, etc.
Upsides are: - It works. The results you are working with remain
consistent, you get notified of any data changes, and you can page
through the results easily. You could even use marker based paging
for the snapshotted results if you wanted to, and it would be
reliable (although some functionality like jumping to an arbitrary
page would be sacrified).
Hope that makes sense, what are everyone's thoughts?
I appreciate that you and others have put a lot of thought and time into
this, but frankly, I just don't see that much benefit to the extra code
complexity and time to deliver it.
Best,
-jay
_______________________________________________
OpenStack-Infra mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
