The following comment has been added to this issue:
Author: Ivan Markov
Created: Mon, 15 Nov 2004 6:59 AM
Body:
Hi James,
I took a brief look your patch. My comments inline (enclosed in []'s).
1) our backend is a mainframe that feeds us data in sets - you cannot know the
full record size without first reading all the sets, which I don't want to have
to do.
[Ivan: OK. But as far as I remember, the DT code needed to know the count of
all records in the full list in order to render the pages' links at the top of
the table. How do you overcome this?]
2) the size of each set is controlled by the backend, not DT. So, although DT
might display 10 records per page, we are forced to get data back in 100-record
sets. 1 data set can back 10 DT pages. This means each page-hit does NOT
coincide with a DB-hit.
[Ivan: Point taken. although such caching scheme can be implemented in specific
DataProvider as well.]
3) the "marker" for fetching the next data set is returned to us by the
previous set - it is not sufficient just to know what page you are on. In order
to get the "marker" for the 10th page, you must first traverse the previous 9.
[Ivan: Again, this could be implemented in the specific DataProvider.]
4) One of the databases we are using is MS SQL Server, which although it does
have the "top" select attribute to get the first 100, it cannot for instance
grab the middle 100. To do this we rely on sorting the data by specific
columns, the specify a ">" where clause to remove the first X records, then use
the top specifier to get the first Y records. This has the overall effect of
getting the middle N records, but seriously limits our sorting capability.
[Ivan: Yes, doing paging with MSSQL is bulky. Still, there is a better scheme
which works fine if your primary key is a single integer column. I can send it
to you if you are interested.]
[Ivan: I still see my solution as the more generic approach. I mean, all
optimizations of yours can be implemented as a specific DataProvider
implementation, can't they? (Don't forget that the DataProvider can be a
stateful object in the user session, so you can cache your 100 records for the
next few page requests.)
The DataProvider thing is the result of my efforts to a) allow the DT code to
push as many time/memory consuming operations as possible to the backend b)
still allow these operations to be implemented in Java if the backend is weak
and can't handle these - I would repeat for the n-th time that for this you
just need a specific DataProvider implementation.
IMO representing the data as iterable list exposes too much semantics to the DT
code and it is too easy to use operations that result in lots of CPU/Network
usage.]
---------------------------------------------------------------------
View this comment:
http://jira.codehaus.org/browse/DISPL-14?page=comments#action_26420
---------------------------------------------------------------------
View the issue:
http://jira.codehaus.org/browse/DISPL-14
Here is an overview of the issue:
---------------------------------------------------------------------
Key: DISPL-14
Summary: Smart Paging
Type: Improvement
Status: Unassigned
Priority: Major
Original Estimate: Unknown
Time Spent: Unknown
Remaining: Unknown
Project: DisplayTag
Components:
Paging/Sorting
Versions:
1.0 RC2
Assignee:
Reporter: fabrizio giustina
Created: Sat, 25 Sep 2004 4:39 AM
Updated: Mon, 15 Nov 2004 6:59 AM
Description:
====
imported from sf tracker
id 1026408
submitted by Ivan Markov - ivan_markov
http://sourceforge.net/tracker/index.php?func=detail&group_id=73068&atid=536613&aid=1026408
====
Smart Paging
------------
Smart paging refers to the ability of DT to deal with large lists of data. The
problem with the current DT approach is that:
a) The whole (potentially large) list of data has to be created for the DT tag,
occupying lots of memory.
b) If the list contains 1000s of records, populated with data from a SQL Server
(99% of the cases, I would say), you have huge network traffic between the
webapp and the server. Our code is especially useful for servers like Oracle &
MySQL which have built-in means to return only a subset of DB cursor's content.
There's one solution + one enhancement on the forums already (issue #1013526),
so why a third one?
We didn't like in the earlier proposal that it deals with intercepting DT's
request and parsing DT's request parameters, which seems a bit hacky. Also, we
avoid the complications of parsing parameters when there is > 1 table tag on
the page. Not to mention that the older proposal does not deal with full list
sorting, or does it?
Instead, we introduced a new interface, DataProvider. It has two methods:
- List getData(int unsortedOffset, int unsortedLength, int recordOffset, int
pageSize, String sortedColumnName, boolean sortOrderAscending);
- int getDataCount();
The first two parameters unsortedOffset & unsortedLength are only needed for
supporting DT's setOffset()/setLength() features.
In our patch, DataProvider is used as a "callback" into user's code. The user
is supposed to implement the two methods of this interface.
The code in TableTag and its supporting classes is changed in a way that, for
each response where DT is rendered,
a) One call is issued to DataProvider.getDataCount(), which retrieves the total
number of rows in the data set (needed for DT to calculate the number of pages).
b) One call is issued to DataProvider.getData(), with the current page, page
size & sorting info.
A sample implementation of user-supplied JDBC DataProvider follows, in
pseudocode. Some JDBC exception handling stuff omitted.
<%
request.setAttribute("list", new DataProvider() {
List getData(int unsortedOffset, int unsortedLength, int recordOffset, int
pageSize, String sortedColumnName, boolean sortOrderAscending) {
Connection con = <app-specific way to get connection>;
PreparedStatement ps = con.prepareStatement("SELECT * FROM MyTable
ORDER BY ? LIMIT ?, ?"):
ps.setString(1, sortedColumnName);
ps.setInt(2, recordOffset);
ps.setInt(3, pageSize);
//etc..
ResultSet rs = con.execute();
return <app-specific way of getting List from ResultSet; Maybe use
RowSet instead of List?>;
}
int getDataCount() {
Connection con = <app-specific way to get connection>;
PreparedStatement ps = con.prepareStatement("SELECT count(*)
FROM MyTable"):
ResultSet rs = con.execute();
rs.next();
return rs.getInt(1);
}
});
%>
(DataProvider implementation code can be moved to Struts Controller/Action,
depending on the MVC framework used, in order not to polute the JSP code with
Java scriptlets.)
Smart Paging Compatibility
--------------------------
We tried to change DT's code so that all older code to continue to work.
If user has provided an old-style List containing all the data, DT will
automatically wrap it with a ListDataProvider.
Current DT functionality (full-list sorting, decorations, multiple tables per
page, etc.) is supported, except for these two cases:
a) Full list(not page) sorting by a decorated column - sorting is done without
regarding the decorator.
b) Full list sorting by a column with no "property" attribute (static or
implicit object call via servlet) - no sorting is performed at all.
---------------------------------------------------------------------
JIRA INFORMATION:
This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
If you want more information on JIRA, or have a bug to report see:
http://www.atlassian.com/software/jira
-------------------------------------------------------
This SF.Net email is sponsored by: InterSystems CACHE
FREE OODBMS DOWNLOAD - A multidimensional database that combines
robust object and relational technologies, making it a perfect match
for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8
_______________________________________________
displaytag-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/displaytag-devel
