You have to consider some things 1) Is this table going to be huge? 2) Am I going to let users do a Full Table Scan? (letting them search without filters, for instance) 3) Is this table frequently updated? How often new data is added to it? 4) Any other question that may affect your scalability ...
Then you will know if is a good idea to just bring the whole data, or do a count(*) before. But, from my experience I rather say: always do a count(*) before. Regards, Bruno -----Mensagem original----- De: Igor Vaynberg [mailto:[EMAIL PROTECTED] Enviada em: quarta-feira, 26 de novembro de 2008 15:20 Para: [email protected] Assunto: Re: Is there any other way? DataProviders must hit the Db twice for (possible) large datasets On Wed, Nov 26, 2008 at 9:06 AM, Wayne Pope <[EMAIL PROTECTED]> wrote: > Hi Igor, > >>what? why would you ever load the whole dataset? > just to avoid 2 calls on smallish datasets, especially when there are > multiple joins and database isnt on the same box. so you think pushing all that extra data over the network is actually more efficient then doing another query???? wtf. >>yeah. because select count() queries are the most expensive queries >>you can run on the database. you are right, its totally going to kill >>it. you know how all those sites on the internet that have a pager >>above the pageable view that shows you the number of the last >>available page...you know how those work without doing a select >>count()? > > Ouch. > I just want to limit calls if possible to the database as waiting for i/o's > is never great for scalability. I'm not 'having a go' at wicket or DataViews > or anything, just trying to understand it. I never claimed to be a guru - > far from it. i can only assume that you have actually profiled your app and that one select count() call was what was taking a significant chunk of processing time in the database server? to the point where eliminating it will actually reduce enough load on the database to increase your throughput? -igor > > Wayne > > > On Wed, Nov 26, 2008 at 5:58 PM, Igor Vaynberg <[EMAIL PROTECTED]>wrote: > >> On Wed, Nov 26, 2008 at 7:32 AM, Wayne Pope >> <[EMAIL PROTECTED]> wrote: >> > I'm sure I must be missing something still, as I can't beleive that we >> need >> > to either a) load the whole data set >> >> what? why would you ever load the whole dataset? >> >> b) call count on the Db , then load in >> > the iterator mehod. Thats going to kill the database in prod (or really >> not >> > help.) >> >> yeah. because select count() queries are the most expensive queries >> you can run on the database. you are right, its totally going to kill >> it. you know how all those sites on the internet that have a pager >> above the pageable view that shows you the number of the last >> available page...you know how those work without doing a select >> count()? >> >> -igor >> >> >> >> >> >> >> > >> > On Wed, Nov 26, 2008 at 3:58 PM, Michael Sparer <[EMAIL PROTECTED] >> >wrote: >> > >> >> >> >> have a look at https://issues.apache.org/jira/browse/WICKET-1784 >> >> >> >> >> >> Wayne Pope-2 wrote: >> >> > >> >> > Ok, >> >> > >> >> > I was just having a bit of code clean up and I realized that in our >> >> > IDataProviders we are loading all rows for a given dataset. >> >> > So looking at the iterator method I see we can limit the result (and >> the >> >> > offset). Great I thought - however I see that that the size() method >> is >> >> > called as part of the getViewSize() in the AbstractPageableView. Thus >> I >> >> > need >> >> > to call the database here to figure out the size. >> >> > >> >> > Am I doing sonething wrong or have I got to hit the database twice for >> >> > each >> >> > DataProvider render. >> >> > >> >> > Obvously I don't want to hard code a size. Is there any other way ? >> >> > >> >> > Thanks >> >> > Wayne >> >> > >> >> > >> >> >> >> >> >> ----- >> >> Michael Sparer >> >> http://talk-on-tech.blogspot.com >> >> -- >> >> View this message in context: >> >> >> http://www.nabble.com/Is-there-any-other-way--DataProviders-must-hit-the-Db-twice-for-%28possible%29-large-datasets-tp20701684p20702476.html >> >> Sent from the Wicket - User mailing list archive at Nabble.com. >> >> >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> >> >> >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] *************************************************************************************************** "Atenção: Esta mensagem foi enviada para uso exclusivo do(s) destinatários(s) acima identificado(s), podendo conter informações e/ou documentos confidencias/privilegiados e seu sigilo é protegido por lei. Caso você tenha recebido por engano, por favor, informe o remetente e apague-a de seu sistema. Notificamos que é proibido por lei a sua retenção, disseminação, distribuição, cópia ou uso sem expressa autorização do remetente. Opiniões pessoais do remetente não refletem, necessariamente, o ponto de vista da CETIP, o qual é divulgado somente por pessoas autorizadas." "Warning: This message was sent for exclusive use of the addressees above identified, possibly containing information and or privileged/confidential documents whose content is protected by law. In case you have mistakenly received it, please notify the sender and delete it from your system. Be noticed that the law forbids the retention, dissemination, distribution, copy or use without express authorization from the sender. Personal opinions of the sender do not necessarily reflect CETIP's point of view, which is only divulged by authorized personnel." *************************************************************************************************** --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
