Re: IDataProvider#size()
On Wed, Apr 11, 2012 at 9:59 PM, Dan Retzlaff wrote: > For SQL-based views, the application complexity comes from the need to > construct a count(*) query with exactly the same criteria as the subsequent > result query. In my experience, this pollutes DAO interfaces and > IDataProvider implementation non-trivially. We initially had separate > methods for counting and querying (same args), but eventually moved to a > single method that returns a -tuple with both the results and > total size which our IDataProvider caches. This lets us do some Hibernate > trickery to introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding > separate count/results queries in most cases. It's still not simple, and > for large counts is still expensive. > > I faced something similar on an earlier project, just using straight JDBC (in Spring), and I ended up taking advice from a book; I made my query SELECT ..., COUNT(*) AS ct FROM ... WHERE ... and I added a ct variable to the class I was mapping to. A lot of extra data was returned, but it didn't slow things too much. I dout this would work well with a NoSQL database, and it is clumsy. Eric
Re: IDataProvider#size()
but if you do not know how many results you have then that can happen using any approach... -igor On Thu, Apr 12, 2012 at 2:36 AM, Michal Wegrzyn wrote: > That's the solution which I have used for the case, where count is not > possible. > It has only one slight disadvantage - if count == n*pageSize then the last > page will be blank. > > Best regards, > Michal Wegrzyn > >> -Original Message- >> From: Igor Vaynberg [mailto:igor.vaynb...@gmail.com] >> Sent: Thursday, April 12, 2012 6:06 >> To: users@wicket.apache.org >> Subject: Re: IDataProvider#size() >> >> why not just fake the size to current page+1? that way you always have >> a "next" link and once you receive the current page you should know if >> you have more or not so you dont have to add the one on the last >> page >> >> -igor >> >> On Wed, Apr 11, 2012 at 6:59 PM, Dan Retzlaff >> wrote: >> > Hi all. Time to start a thread of my own. :) >> > >> > Many of Wicket's powerful repeaters depend on IDataProvider. This >> > interface has a size() method which returns a non-null integer. This >> > makes it easy to determine the total number of pages in a pageable >> > view, but IMO the required computation and application complexity are >> not always called for. >> > In many cases, a pageable but open-ended data view is adequate. Have >> > you experienced this impedance mismatch yourselves? What was your >> solution? >> > >> > To elaborate on my experience: >> > >> > For SQL-based views, the application complexity comes from the need >> to >> > construct a count(*) query with exactly the same criteria as the >> > subsequent result query. In my experience, this pollutes DAO >> > interfaces and IDataProvider implementation non-trivially. We >> > initially had separate methods for counting and querying (same args), >> > but eventually moved to a single method that returns a >> > -tuple with both the results and total size which our >> > IDataProvider caches. This lets us do some Hibernate trickery to >> > introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding separate >> > count/results queries in most cases. It's still not simple, and for >> large counts is still expensive. >> > >> > The situation is worse for non-SQL data stores which don't have a >> > fully-functional count(*) capability. We use Cassandra whose native >> > "where clause" support is limited, requiring significant client-side >> filtering. >> > Paging through an entire column (or CF) in this way is prohibitively >> > expensive, especially considering our users rarely even go to page 2. >> > To solve this, we've created a parallel set of view/paging classes >> > that define windows using previously discovered result keys instead >> of >> > start indices (tokens and column names in Cassandra). But having a >> > full suite of IUnsizedDataProvider-based classes smells. I love that >> > Wicket devs have solved some tough/tedious problems with DataViewBase >> > and friends, and I want to make use of them! >> > >> > Comments or suggestions? >> > >> > Cheers, >> > Dan >> >> - >> To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org >> For additional commands, e-mail: users-h...@wicket.apache.org > > > - > To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org > For additional commands, e-mail: users-h...@wicket.apache.org > - To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org For additional commands, e-mail: users-h...@wicket.apache.org
RE: IDataProvider#size()
That's the solution which I have used for the case, where count is not possible. It has only one slight disadvantage - if count == n*pageSize then the last page will be blank. Best regards, Michal Wegrzyn > -Original Message- > From: Igor Vaynberg [mailto:igor.vaynb...@gmail.com] > Sent: Thursday, April 12, 2012 6:06 > To: users@wicket.apache.org > Subject: Re: IDataProvider#size() > > why not just fake the size to current page+1? that way you always have > a "next" link and once you receive the current page you should know if > you have more or not so you dont have to add the one on the last > page > > -igor > > On Wed, Apr 11, 2012 at 6:59 PM, Dan Retzlaff > wrote: > > Hi all. Time to start a thread of my own. :) > > > > Many of Wicket's powerful repeaters depend on IDataProvider. This > > interface has a size() method which returns a non-null integer. This > > makes it easy to determine the total number of pages in a pageable > > view, but IMO the required computation and application complexity are > not always called for. > > In many cases, a pageable but open-ended data view is adequate. Have > > you experienced this impedance mismatch yourselves? What was your > solution? > > > > To elaborate on my experience: > > > > For SQL-based views, the application complexity comes from the need > to > > construct a count(*) query with exactly the same criteria as the > > subsequent result query. In my experience, this pollutes DAO > > interfaces and IDataProvider implementation non-trivially. We > > initially had separate methods for counting and querying (same args), > > but eventually moved to a single method that returns a > > -tuple with both the results and total size which our > > IDataProvider caches. This lets us do some Hibernate trickery to > > introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding separate > > count/results queries in most cases. It's still not simple, and for > large counts is still expensive. > > > > The situation is worse for non-SQL data stores which don't have a > > fully-functional count(*) capability. We use Cassandra whose native > > "where clause" support is limited, requiring significant client-side > filtering. > > Paging through an entire column (or CF) in this way is prohibitively > > expensive, especially considering our users rarely even go to page 2. > > To solve this, we've created a parallel set of view/paging classes > > that define windows using previously discovered result keys instead > of > > start indices (tokens and column names in Cassandra). But having a > > full suite of IUnsizedDataProvider-based classes smells. I love that > > Wicket devs have solved some tough/tedious problems with DataViewBase > > and friends, and I want to make use of them! > > > > Comments or suggestions? > > > > Cheers, > > Dan > > - > To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org > For additional commands, e-mail: users-h...@wicket.apache.org - To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org For additional commands, e-mail: users-h...@wicket.apache.org
Re: IDataProvider#size()
On Thu, Apr 12, 2012 at 9:12 AM, Martin Grigorov wrote: > On Thu, Apr 12, 2012 at 4:59 AM, Dan Retzlaff wrote: >> Hi all. Time to start a thread of my own. :) >> >> Many of Wicket's powerful repeaters depend on IDataProvider. This interface >> has a size() method which returns a non-null integer. This makes it easy to >> determine the total number of pages in a pageable view, but IMO the >> required computation and application complexity are not always called for. >> In many cases, a pageable but open-ended data view is adequate. Have you >> experienced this impedance mismatch yourselves? What was your solution? > > Wicketstuff InMethod Grid supports this use case. > One of its examples shows it. > I'll deploy the examples at wicket-library.com soon. http://www.wicket-library.com/inmethod-grid/data-grid/unknown-count > >> >> To elaborate on my experience: >> >> For SQL-based views, the application complexity comes from the need to >> construct a count(*) query with exactly the same criteria as the subsequent >> result query. In my experience, this pollutes DAO interfaces and >> IDataProvider implementation non-trivially. We initially had separate >> methods for counting and querying (same args), but eventually moved to a >> single method that returns a -tuple with both the results and >> total size which our IDataProvider caches. This lets us do some Hibernate >> trickery to introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding >> separate count/results queries in most cases. It's still not simple, and >> for large counts is still expensive. >> >> The situation is worse for non-SQL data stores which don't have a >> fully-functional count(*) capability. We use Cassandra whose native "where >> clause" support is limited, requiring significant client-side filtering. >> Paging through an entire column (or CF) in this way is prohibitively >> expensive, especially considering our users rarely even go to page 2. To >> solve this, we've created a parallel set of view/paging classes that define >> windows using previously discovered result keys instead of start indices >> (tokens and column names in Cassandra). But having a full suite of >> IUnsizedDataProvider-based classes smells. I love that Wicket devs have >> solved some tough/tedious problems with DataViewBase and friends, and I >> want to make use of them! >> >> Comments or suggestions? >> >> Cheers, >> Dan > > > > -- > Martin Grigorov > jWeekend > Training, Consulting, Development > http://jWeekend.com -- Martin Grigorov jWeekend Training, Consulting, Development http://jWeekend.com - To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org For additional commands, e-mail: users-h...@wicket.apache.org
Re: IDataProvider#size()
On Thu, Apr 12, 2012 at 4:59 AM, Dan Retzlaff wrote: > Hi all. Time to start a thread of my own. :) > > Many of Wicket's powerful repeaters depend on IDataProvider. This interface > has a size() method which returns a non-null integer. This makes it easy to > determine the total number of pages in a pageable view, but IMO the > required computation and application complexity are not always called for. > In many cases, a pageable but open-ended data view is adequate. Have you > experienced this impedance mismatch yourselves? What was your solution? Wicketstuff InMethod Grid supports this use case. One of its examples shows it. I'll deploy the examples at wicket-library.com soon. > > To elaborate on my experience: > > For SQL-based views, the application complexity comes from the need to > construct a count(*) query with exactly the same criteria as the subsequent > result query. In my experience, this pollutes DAO interfaces and > IDataProvider implementation non-trivially. We initially had separate > methods for counting and querying (same args), but eventually moved to a > single method that returns a -tuple with both the results and > total size which our IDataProvider caches. This lets us do some Hibernate > trickery to introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding > separate count/results queries in most cases. It's still not simple, and > for large counts is still expensive. > > The situation is worse for non-SQL data stores which don't have a > fully-functional count(*) capability. We use Cassandra whose native "where > clause" support is limited, requiring significant client-side filtering. > Paging through an entire column (or CF) in this way is prohibitively > expensive, especially considering our users rarely even go to page 2. To > solve this, we've created a parallel set of view/paging classes that define > windows using previously discovered result keys instead of start indices > (tokens and column names in Cassandra). But having a full suite of > IUnsizedDataProvider-based classes smells. I love that Wicket devs have > solved some tough/tedious problems with DataViewBase and friends, and I > want to make use of them! > > Comments or suggestions? > > Cheers, > Dan -- Martin Grigorov jWeekend Training, Consulting, Development http://jWeekend.com - To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org For additional commands, e-mail: users-h...@wicket.apache.org
Re: IDataProvider#size()
why not just fake the size to current page+1? that way you always have a "next" link and once you receive the current page you should know if you have more or not so you dont have to add the one on the last page -igor On Wed, Apr 11, 2012 at 6:59 PM, Dan Retzlaff wrote: > Hi all. Time to start a thread of my own. :) > > Many of Wicket's powerful repeaters depend on IDataProvider. This interface > has a size() method which returns a non-null integer. This makes it easy to > determine the total number of pages in a pageable view, but IMO the > required computation and application complexity are not always called for. > In many cases, a pageable but open-ended data view is adequate. Have you > experienced this impedance mismatch yourselves? What was your solution? > > To elaborate on my experience: > > For SQL-based views, the application complexity comes from the need to > construct a count(*) query with exactly the same criteria as the subsequent > result query. In my experience, this pollutes DAO interfaces and > IDataProvider implementation non-trivially. We initially had separate > methods for counting and querying (same args), but eventually moved to a > single method that returns a -tuple with both the results and > total size which our IDataProvider caches. This lets us do some Hibernate > trickery to introduce a MySQL SQL_CALC_FOUND_ROWS query hint, avoiding > separate count/results queries in most cases. It's still not simple, and > for large counts is still expensive. > > The situation is worse for non-SQL data stores which don't have a > fully-functional count(*) capability. We use Cassandra whose native "where > clause" support is limited, requiring significant client-side filtering. > Paging through an entire column (or CF) in this way is prohibitively > expensive, especially considering our users rarely even go to page 2. To > solve this, we've created a parallel set of view/paging classes that define > windows using previously discovered result keys instead of start indices > (tokens and column names in Cassandra). But having a full suite of > IUnsizedDataProvider-based classes smells. I love that Wicket devs have > solved some tough/tedious problems with DataViewBase and friends, and I > want to make use of them! > > Comments or suggestions? > > Cheers, > Dan - To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org For additional commands, e-mail: users-h...@wicket.apache.org