RE: Is there any other way? DataProviders must hit the Db twice for (possible) large datasets

[rgoodwin]

Still is being disputed unfortunately :(

https://issues.apache.org/jira/browse/WICKET-1784

For those who believe in the 1-query approach, ergo the DTO pattern, writing
an alternative to the Wicket data provision using a DTO approach isn't a
terrible hardship ... just annoying to be sprung with on day one of using
Wicket. But pain is experienced later when encountering the Wicket data
provision and paging mechanism coupling. The paging mechanism is far less
joy to replace ...

IReallyHateDataProvider



Hoover, William wrote:
> 
> I think the idea behind this is that size will be called first. If the
> size is zero there is no need to proceed with the call to get the items.
> I don't necessarily agree with this approach because a lot of service
> calls can capture the data in one call (even down to the database level-
> some support getting the size/results in one call), but the last time I
> brought this issue up it was disputed.
> 

-- 
View this message in context: 
http://www.nabble.com/Is-there-any-other-way--DataProviders-must-hit-the-Db-twice-for-%28possible%29-large-datasets-tp20701684p20708929.html
Sent from the Wicket - User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Paging query relating to IDataProvider and Lucene search

[rgoodwin]

Hi John,

Thanks for replying. Would be a neat solution if only I could cache Hits.
:(

But in my desire to simplify the problem statement I omitted to mention that
our data services tier runs in another cluster and converts Lucene
hits/documents/fields to a collection of our in-house search results
objects. No Lucene objects are sent back to the UI server.

These search result objects are a bit like DTOs, but not in the pre-EJB3
sense. They're really just to standardise our search results architecture,
as certain search result subclasses actually do db hits. So we want
everything unified for the web app.

This is why I made reference in the Wicket Jira ticket WICKET-1784 to the
Transfer Object pattern, as solving this problem with respect to DTOs
(instead of just Lucene) would solve for all interaction with legacy systems
using DTOs, me thinks.

(DTOs get a bad name these days now that entities are detachable, but I
believe they still have their uses, e.g. for abstracting)

Thanks again, and very open to simple solutions.
Not keen on re-inventing the Wicket wheel.
-- 
View this message in context: 
http://www.nabble.com/Paging-query-relating-to-IDataProvider-and-Lucene-search-tp19083164p19094471.html
Sent from the Wicket - User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Paging query relating to IDataProvider and Lucene search

[rgoodwin]

Hello,

I'm trying to solve a variation on a standard problem regarding running
searches for paged results and displaying paging links.

Unfortunately for me, I cannot use the standard PagingNavigator / DataView /
IDataProvider solution for this. Wicket's top-down approach to processing
has a consequence in that there must be separate queries run for size and
data.

But our current web app retrieves its data from Lucene search results.
A Lucene query returns both results and size simultaneously.
Only 1 query is performed.


The paging related and DataView are calling IDataProvider.size() early on,
before calling iterator() and passing the offset. Of course this fits
perfectly for most users and I see from the code why it has to be done like
this. But this also seems to conflict with a data service tier expecting to
perform a single query to return both results ans size in one, e.g. like
Lucene or when using a Transfer Object pattern.

Is there any easy way to subclass either PagingNavigator or associated
classes so that they don't get involved with IPageable until after all data
is loaded?


---

Not as weird a suggestion as you might think.
Having paging defer it's configuration until last is exactly how our current
servlet/JSP app works.
So no need to get hold of size before querying for paged results.

1. User clicks paging link which sends an offset value
2. Server performs Lucene search which returns size and hits
3. A paged subset of results starting at offset are made ready for
presentation
4. Server creates Paging class with the latest offset and size etc
5. Server hands over to JSP
6. JSP renders results and renders paging links


Wicket sequence seems approximately the opposite:

1. User clicks paging link which sends page number
2. PagingNavigationLink calls its IPageable.getPageCount()
3. IPageable then calls IDataProvider.size()
4. Query 1 is run to get size
3. DataView calls IDataProvider to get size()
4. DataView then calls iterator() to get results
5. Query 2 is run to get results
4. Then page rendering starts?

With only a basic understanding of the Wicket api, it seems to me data
loading and component configuration stages are interwoven?


I already wrote an alternative to IDataProvider and DataView that supports
single query for size and data,
and posted the code for this to Jira ticket WICKET-1784.

Would now like to develop an alternative paging component to solve the
problem completely, in that the paging behaves like our current solution,
e.g. it 'reacts' to the latest data after it's loaded, rather than try to
predict it.


Many thanks.
-- 
View this message in context: 
http://www.nabble.com/Paging-query-relating-to-IDataProvider-and-Lucene-search-tp19083164p19083164.html
Sent from the Wicket - User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]