What I have seen done is to have an object store the query information,
including sorting options, in an object, along with a transient (so that
during failover, it gets regenerated) index, and a NON-transient
include/exclude selection list (ie either items that are included are
listed, or items that are excluded are listed, depending on how the UI
buttons are used.) The index is typically an array of long primary key
values. Other PK types are possible, but longs are by far the most common.
Then, on each page request, a query is made of the form:
select X from Y where Y.PK in {a,b,c,d,e,f,...}
This query can either be generated dynamically or kept as a transient
prepared statement.
If this is done carefully, then the data can come from a variety of
sources, and can have plugin interfaces for the primary key list and the
paging information. Then, taglibs can be used of the form:
<tr> <pageHeader name="pagedObjectName" allowSort="true" column="X">Column
Header</pageHeader> </tr>
<iterate name="pagedObjectName" attribute="pageIterator" child="lineItem">
<tr> <td><getProperty name="lineItem" property="x"></td> </tr>
</iterate>
<nextButton name="pagedObjectName" /> <prevButton name="pagedObjectName"
/>
With a very few custom tags, this can be used to generate paging quite
effectively, while allowing selection and sorting. Note that the sorting
requires re-generating the index list, but does not require regenrating the
selection list.
As well, for very large lists, the index generator method can be replaced
with a custom generator that pages the index list as well as the sub pages
(ie get at most 5000 hits at once, page through 100 of those at a time,
when "next" is selected, at the 5000 page boundary, load the next 5000
items.)
This system will work either with direct DB access, or going through EJB's.
I don't recommend EJB's for a high performance system, but it works fine
for lower volumes, or for initial proof of concept. For higher volumes,
it can still work with EJB's if the EJB implements a bulk get/set operation
instead of individual item operations, or if one carefully writes the EJB
code in such a fashion as to allow access either via the EJB, or via custom
calls that create the select statements and generate return elements (this
second method was how I have seen it done - it worked very well, and
ensured that there was only one way to load any given value object into
memory, albeit via two different interfaces.)
I have seen systems that load all the data into memory, and it is very
slow. I've also seen ones that load on each page call. The problem with
those ones is that getting further into the list becomes slower and slower.
As well, going forward/back doesn't necessarily return the same page. It
also doesn't support end very well. The system presented here works
extremely well, both for paged and non-paged data, is relatively fast, and
interoperates with existing tags extremely well. Even when used without
tags, it still works quite nicely. Unfortunately, I don't have an
implementation of it to donate. I might write one within a couple of
months, but I'm still discussing with my boss about when/how to contribute
internal source for external use.
Bill Wallace
(716) 214-8872
Fax: (716) 295-4718
Element K
'the knowledge catalyst'
www.elementk.com
"Brian
Cochran" To: "Struts Developers List"
<brian@bricom <[EMAIL PROTECTED]>
.com> cc:
11/15/01 Subject: Other possible tools for paging
06:31 PM collections
Please
respond to
"Struts
Developers
List"
Writing paging (i.e. next, previous, etc) iterators can be a real pain to
do
correctly (at least it is for me).
Looking at the petstore, they seem to wuss out on doing it nice. It
occured
to me that there are a few options as far as doing iteration over pages.
1) retrieve all results and keep them in memory (in a session var i guess)
while the user pages through them.
2) go to the db every time.
3) JDBC 2.0 allows scrollable recordsets so you could have time-out-able
recordsets that essentially close themselves
on a time expiration and may reopen themselves if needed. (May be some
nasty concurrency problems with this)
4) create a hybrid that will cache some of the pages and go to the database
if needed. Maybe be able to configure
the number of pages to cache. Really cool would be able to change it
dynamically.
5) somebody elses neat idea
But whatever is used there is the basic principle of getEntities(int index,
int span). So what do you guys think about
a series of paging iterators that looked something like this.
abstract class PagingIterator
abstract List getEntities(int startIndex, int span)
int getPage()
void setPage(int page)
boolean hasNextPage()
void nextPage()
boolean hasPreviousPage()
void previousPage()
class RequeryPagingIterator extends PagingIterator // like the petstore
abstract List getEntities(int startIndex, int span)
class CacheingPagingIterator extends PaginingIterator // good for a lot
of smaller datasets
List _AllEntities = null;
List getEntities(int startIndex, int span){
if(_AllEntities == null){
getAllEntities();
}
int stopIndex = max(startIndex + span, _AllEntities.size() -
1);
return _AllEntites.subList(startIndex, stopIndex);
}
class MultiplePageCachingPagingIterator extends PagingIterator // great
little combination
...
class ResultSetPagingIterator extends PagingIterator { // little new to
JDBC 2 so not sure bout this (suggestions)
... // used with JDBC 2.0
}
The cool thing is the tags on the JSP's all work with PagingIterator so
there is no need for the page to know how
the paging is implemented.
Some other things. Currently i have a messy implementation of this running
inside struts. Works ok. but basically some
of the challenges include
1) knowing when the query is new and when its navigation. currently I
check
for the presense of a navigate in
the request params.
2) where and when to stash the iterator. Currently I'm stashing it in the
session using some other custom tags.
3) should the getEntities thing be done as an interface instead of an
abstract method? e.g
new SomePagingIterator(ChunkableRetriever c) where
ChunkableRetriever has
a method getEntities(int start, int span)
3) how do you cleanly handle errors (I suck at error handeling in JSPs)
Here's an example how its being used.
ItemsByCategoryPagingIterator extends RequeryPagingIterator {
String _CategoryId = null;
Collection getEntities(int start, int span) throws SomeException {
Collection col = null;
try {
col = itemsHome.findByCategoryId(getCategoryId());
}catch(BusinessException e){
throw new SomeException(e.getMessage());
}
return col;
}
}
Then i have some definition code that initiates sets all the properties
(categoryId in the above example)
and then sends it off to the jsp to do its navigation.
There is also
<list:chunk iterator="wheretheiteratorisstashed" collectionTarget="items"
navigate="<%= navoption from request %>" >
<logic:iterate id="eachitem" name="items">
present the properties of eachitem
</logic:iterate>
<list:previous>something to display if there is a previous
(link)</list:previous>
<list:next>something to display if there is a next (link) </list:next>
</list:chunk>
Disclaimer:
I am not an architecht. Moreover, I am just out of college and still wet
behind the ears, but I would be glad to help
out with this. I would love to hear what you guys think.
Thanks a bunch in advance and keep up the great work.
Brian
--
To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]
>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]
>
--
To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
