Bugs item #1023387, was opened at 2004-09-07 11:40
Message generated for change (Comment added) made by ianbdev
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=536613&aid=1023387&group_id=73068
Category: main tag library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Ian Barnett (ianbdev)
Assigned to: Nobody/Anonymous (nobody)
Summary: Pagination - use subList instead of iterating all records
Initial Comment:
The code that obtains a subset of records from the List
object (which is the subject of the display table)
iterates through all the records in the list until it finds
the ones it requires.
This is no problem for small lists where the subset might
be page 2 as 21-40 for example as only 20 records are
skipped during iteration.
But when the lists are very large (for example 20,000
records) and the page is a high numbered page (e.g.
page 9001, records 18001-18021) the iteration is very
expensive as it must iterate through and skip over
18,000 records just to get to the first record.
We have a simple modified verision of SmartListHelp.java
(based on the 1.0rc-1 library) that does not iterate
through unwanted records but rather uses the List
interface's subList method to retrieve only the records
required.
Then it is up to the application developer to either
burden the list with all (for example 20,000) records or
implement a List class that provides a subList method
that intelligently retrieves only the records required from
the List's datasource (e.g. may only grab 20 records at
a time for the database).
This is a very lightweight solution to the calls for
improved long list pagination that has a low impact on
the Display Tag code base.
Below is the current code for paginated record retrieval:
/**
* Returns a subsection of the list that contains just
the elements that are supposed to be shown on the
given page.
* @param pageNumber page number
* @return List subsection of the list that contains
just the elements that are supposed to be shown on the
given
* page
*/
protected List getListForPage(int pageNumber) {
log.debug("getListForPage page=" + pageNumber);
List list = new ArrayList(this.pageSize + 1);
int firstIndex = getFirstIndexForPage(pageNumber);
int lastIndex = getLastIndexForPage(pageNumber);
Iterator iterator = this.fullList.iterator();
int j = 0;
while (iterator.hasNext()) {
Object object = iterator.next();
if (j > lastIndex) {
break;
} else if (j >= firstIndex) {
list.add(object);
}
j++;
}
return list;
}
The recommended fix is:
protected List getListForPage(int pageNumber) {
log.debug("getListForPage page=" + pageNumber);
int firstIndex = getFirstIndexForPage(pageNumber);
int lastIndex = getLastIndexForPage(pageNumber);
return fullList.subList(firstIndex, lastIndex+1);
}
----------------------------------------------------------------------
>Comment By: Ian Barnett (ianbdev)
Date: 2004-09-13 09:08
Message:
Logged In: YES
user_id=461701
The value list handler takes care of this by only ever loading
the number of records you need to display. There is an initial
database query to get the size of the list, after that each
page of records only is loaded from the database not the
whole lot. See RFE 872183 for details on how to do the
database queries.
After posting this and attempting to test I found that we
were hampered by other elements of the tag library that
made assumptions about the list being fully loaded. As a
result we had to make further changes. I also had based
changes against 1.0b2 not 1.0rc1, my apologies for any
confusion.
Attached are the two files (draft only) we had to modify
(from 1.0b2) to get this working. Please note there are some
drastic assumptions we have made to get this working.
Sorting does not work at the list level when paging at the
database level as one would expect (dynamic sorting at the
database query is required for this).
The displaytag code also seems to make allowances for
iterable objects other than the List interface and for the
purpose of this change we had to assume a List interface (for
now).
I will continue to work on our proposed solution and do
another post soon preferably against the latest display tag
code base.
Please note however there will always be caveats against this
method of pagination specifically in regard to the assumptions
made concerning the availability of the whole list (i.e.
exporting, sorting, target list interface type).
Additional TableTag changes...
Changed:
private void initParameters() throws ObjectLookupException
To:
private void initParameters() throws
ObjectLookupException, JspException
Added to the end of initParameters():
// If apparently paging then iterate only over the page
(sorting at page level only now)
List listAsList = (List)this.list;
List pageData = getRequiredViewList(listAsList);
this.tableModel.setRowListPage(pageData);
if (listAsList.size() != pageData.size()) {
this.tableIterator = IteratorUtils.getIterator
(pageData);
}
Changed:
List pageData = getViewableData();
To:
List pageData = this.tableModel.getRowListFull();
Changed:
Object originalData = this.tableModel.getRowListFull();
To:
List originalData = this.tableModel.getRowListFull();
Changed (line 1037):
fullList = CollectionUtil.getListFromObject(originalData,
this.offset, this.length);
To:
// Use subList to get range (assumes List interface for
now)
// fullList = CollectionUtil.getListFromObject
(originalData, this.offset, this.length);
if (this.offset!=0 && this.length!=0) {
fullList = originalData.subList(this.offset,
this.offset+this.length);
} else {
fullList = originalData;
}
Added:
private List getRequiredViewList(List originalList) throws
JspException
{
log.debug("getRequiredViewList called.");
final List tempList = getOffsetRange(originalList,
this.offset, this.length);
// If they have asked for just a page of the data, then
use the
// SmartListHelper to figure out what page they are
after, etc...
log.debug("getRequiredViewList - limit to page,
pagesize=:"+this.pagesize);
if (this.pagesize > 0) {
this.listHelper = new SmartListHelper(tempList,
this.pagesize, this.properties);
this.listHelper.setCurrentPage(this.pageNumber);
return this.listHelper.getListForCurrentPage();
} else {
return tempList;
}
}
Added:
private List getOffsetRange(List originalList, int offset, int
length) {
// If they have asked for a subset of the list via the
length
// attribute, then only fetch those items out of the
master list.
if (offset!=0 && length!=0) {
return originalList.subList(offset, offset+length);
} else {
return originalList;
}
}
----------------------------------------------------------------------
Comment By: Nobody/Anonymous (nobody)
Date: 2004-09-10 18:26
Message:
Logged In: NO
However,if a project has 100000 records,when first loading all
the records from database,the speed would be seriously
influenced by using this method.how can i do this?
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=536613&aid=1023387&group_id=73068
-------------------------------------------------------
This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170
Project Admins to receive an Apple iPod Mini FREE for your judgement on
who ports your project to Linux PPC the best. Sponsored by IBM.
Deadline: Sept. 13. Go here: http://sf.net/ppc_contest.php
_______________________________________________
displaytag-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/displaytag-devel
