>> you mentioned that you implemented such a repeater yourself. didn't you use >> any navigation or did you write that yourself? just wondering.
>i implemented a simple prev/next pager which is all that was required >or that usecase. Matej's implementation from inMethods works perfectly. I suggest you use that Stefan if you need such a thing. (Thanks Matej!) On Thu, Nov 27, 2008 at 7:51 PM, Igor Vaynberg <[EMAIL PROTECTED]>wrote: > On Thu, Nov 27, 2008 at 10:28 AM, Stefan Fußenegger > <[EMAIL PROTECTED]> wrote: > > > > Hi Igor, > > > > thanks for implementing this minimal version. i totally agree with your > > reasoning. Is there any chance though that this goes into 1.3 branch as > > well? I'd really appreciate that. > > done > > > you mentioned that you implemented such a repeater yourself. didn't you > use > > any navigation or did you write that yourself? just wondering. > > i implemented a simple prev/next pager which is all that was required > for that usecase. > > > shall i open a ticket against 1.5 to track this issue/enhancement? > > i added it to the wishlist wiki page. if you add a jira ticket please > add the number to the wiki page. > > -igor > > > > > best regards, stefan > > > > > > > > igor.vaynberg wrote: > >> > >> On Thu, Nov 27, 2008 at 12:46 AM, Stefan Fußenegger > >> <[EMAIL PROTECTED]> wrote: > >>> > >>> I don't think IDataProvider is only about databases. > >> > >> you started off with your core assumption being wrong. idataprovider > >> was written exclusively for accessing databases. my thinking, at the > >> time, was that 99% of people use wicket to build applications that > >> access databases, and i dare say it was a good guess because in its ~3 > >> years of existence only a handful of people had a problem with the way > >> it works. > >> > >>> There are other data > >>> sources and some return the total amount and the desired subset at the > >>> same > >>> time (Lucene as a famous example). Such data sources would really > benefit > >>> of > >>> a single-query-approach. > >> > >> i am not disputing this fact. i am simply saying that we are not going > >> to fix this right now because this is not a bug. you are trying to use > >> the components for something they were not designed to be used. in 1.5 > >> we may address this. > >> > >>> I faced this issue myself in a search (read Lucene) > >>> centered application. I successfully went down the road of implementing > a > >>> custom repeater. > >> > >> i had to do the same myself. > >> > >>> But when the repeater was working as desired, I figured out > >>> that PagingNavigationLink is the real showstopper, not IDataProvider > (see > >>> my > >>> JIRA comment [0]). The fix would be rather trivial, as > >>> PagingNavigationLink > >>> is doing something it needn't do (checking the requested page against > the > >>> valid range of pages). > >>> > >>> Let me answer 2 possible questions in advance: > >>> > >>> Q: Why is this check in PagingNavigationLink a problem? > >>> A: Obviously, you can't fetch size and data as long as the page isn't > >>> known. > >> > >> the check is there because we code defensively. we do not assume that > >> every implementation of ipageable will cull the number when you call > >> setcurrentpage(x). > >> > >>> Q: How would a custom repeater that fetches data and size at the same > >>> tame > >>> handle invalid (out of range) pages? > >>> A: Out of range pages will return the size and an empty dataset. In > this > >>> case, the repeater would change the page number to the last valid and > do > >>> a > >>> second query. Yeah, two queries again. But this should only happen > rarely > >>> though. > >> > >> this will change the existing behavior. if you are on page 5 and click > >> page 10 (which happens to not exist) you would end up back on 5 with > >> your suggestion where as currently you would properly end up on 9. > >> > >> looking at WICKET-1784, i extracted the code you want into an > >> overridable int cullPageNumber(int x). so feel free to subclass the > >> link and override that to return x without any extra checks. > >> > >> we may properly fix this in 1.5, but for right now this is too big a > >> refactor because it changes the basic assumptions with which the code > >> was written. > >> > >> -igor > >> > >>> > >>> Best regards, Stefan > >>> > >>> [0] > >>> > https://issues.apache.org/jira/browse/WICKET-1784?focusedCommentId=12651278#action_12651278 > >>> > >>> > >>> igor.vaynberg wrote: > >>>> > >>>> On Wed, Nov 26, 2008 at 9:32 AM, Wayne Pope > >>>> <[EMAIL PROTECTED]> wrote: > >>>>>>so you think pushing all that extra data over the network is actually > >>>>>>more efficient then doing another query???? wtf. > >>>>> The point is I'd rather avoid 2 calls where 1 will do. > >>>>> AbstractPageableView > >>>>> will do fine I believe. > >>>> > >>>> the number of calls itself is meaningless, i dont comprehend why > >>>> people have a hard time understanding this simple fact. > >>>> > >>>> if you have one call that takes 1000ms and ten calls that each take > >>>> 10ms you should concentrate on the one call that takes a long time > >>>> rather then eliminating all ten 10ms calls which only saves you 100ms. > >>>> if you can optimize the 1000ms and shave off 20% then your eleven > >>>> calls are still faster then the one call. > >>>> > >>>> and since connection pools have been inventind many years ago there is > >>>> no more overhead of establishing network connections, just pushing > >>>> bits around. maybe that is still a problem in php, but in java it has > >>>> been solved a long time ago. > >>>> > >>>> -igor > >>>> > >>>> > >>>>> > >>>>>>i can only assume that you have actually profiled your app and that > >>>>>>one select count() call was what was taking a significant chunk of > >>>>>>processing time in the database server? to the point where > eliminating > >>>>>>it will actually reduce enough load on the database to increase your > >>>>>>throughput? > >>>>> > >>>>> No I haven't, as mentioned before, I just want to avoid 2 calls when > >>>>> one > >>>>> will do. I have however seen several times in production systems > >>>>> waiting > >>>>> on > >>>>> i/o's reduces your scalability. I'd rather keep server count down as > >>>>> money > >>>>> is tight. > >>>>> I'll be mindfull not to ask 'stupid' questions again. > >>>>> > >>>>> > >>>>> > >>>>> On Wed, Nov 26, 2008 at 6:19 PM, Igor Vaynberg > >>>>> <[EMAIL PROTECTED]>wrote: > >>>>> > >>>>>> On Wed, Nov 26, 2008 at 9:06 AM, Wayne Pope > >>>>>> <[EMAIL PROTECTED]> wrote: > >>>>>> > Hi Igor, > >>>>>> > > >>>>>> >>what? why would you ever load the whole dataset? > >>>>>> > just to avoid 2 calls on smallish datasets, especially when there > >>>>>> are > >>>>>> > multiple joins and database isnt on the same box. > >>>>>> > >>>>>> so you think pushing all that extra data over the network is > actually > >>>>>> more efficient then doing another query???? wtf. > >>>>>> > >>>>>> >>yeah. because select count() queries are the most expensive > queries > >>>>>> >>you can run on the database. you are right, its totally going to > >>>>>> kill > >>>>>> >>it. you know how all those sites on the internet that have a pager > >>>>>> >>above the pageable view that shows you the number of the last > >>>>>> >>available page...you know how those work without doing a select > >>>>>> >>count()? > >>>>>> > > >>>>>> > Ouch. > >>>>>> > I just want to limit calls if possible to the database as waiting > >>>>>> for > >>>>>> i/o's > >>>>>> > is never great for scalability. I'm not 'having a go' at wicket or > >>>>>> DataViews > >>>>>> > or anything, just trying to understand it. I never claimed to be a > >>>>>> guru - > >>>>>> > far from it. > >>>>>> > >>>>>> i can only assume that you have actually profiled your app and that > >>>>>> one select count() call was what was taking a significant chunk of > >>>>>> processing time in the database server? to the point where > eliminating > >>>>>> it will actually reduce enough load on the database to increase your > >>>>>> throughput? > >>>>>> > >>>>>> -igor > >>>>>> > >>>>>> > > >>>>>> > Wayne > >>>>>> > > >>>>>> > > >>>>>> > On Wed, Nov 26, 2008 at 5:58 PM, Igor Vaynberg > >>>>>> <[EMAIL PROTECTED] > >>>>>> >wrote: > >>>>>> > > >>>>>> >> On Wed, Nov 26, 2008 at 7:32 AM, Wayne Pope > >>>>>> >> <[EMAIL PROTECTED]> wrote: > >>>>>> >> > I'm sure I must be missing something still, as I can't beleive > >>>>>> that > >>>>>> we > >>>>>> >> need > >>>>>> >> > to either a) load the whole data set > >>>>>> >> > >>>>>> >> what? why would you ever load the whole dataset? > >>>>>> >> > >>>>>> >> b) call count on the Db , then load in > >>>>>> >> > the iterator mehod. Thats going to kill the database in prod > (or > >>>>>> really > >>>>>> >> not > >>>>>> >> > help.) > >>>>>> >> > >>>>>> >> yeah. because select count() queries are the most expensive > queries > >>>>>> >> you can run on the database. you are right, its totally going to > >>>>>> kill > >>>>>> >> it. you know how all those sites on the internet that have a > pager > >>>>>> >> above the pageable view that shows you the number of the last > >>>>>> >> available page...you know how those work without doing a select > >>>>>> >> count()? > >>>>>> >> > >>>>>> >> -igor > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > > >>>>>> >> > On Wed, Nov 26, 2008 at 3:58 PM, Michael Sparer < > >>>>>> [EMAIL PROTECTED] > >>>>>> >> >wrote: > >>>>>> >> > > >>>>>> >> >> > >>>>>> >> >> have a look at > https://issues.apache.org/jira/browse/WICKET-1784 > >>>>>> >> >> > >>>>>> >> >> > >>>>>> >> >> Wayne Pope-2 wrote: > >>>>>> >> >> > > >>>>>> >> >> > Ok, > >>>>>> >> >> > > >>>>>> >> >> > I was just having a bit of code clean up and I realized that > >>>>>> in > >>>>>> our > >>>>>> >> >> > IDataProviders we are loading all rows for a given dataset. > >>>>>> >> >> > So looking at the iterator method I see we can limit the > >>>>>> result > >>>>>> (and > >>>>>> >> the > >>>>>> >> >> > offset). Great I thought - however I see that that the > size() > >>>>>> method > >>>>>> >> is > >>>>>> >> >> > called as part of the getViewSize() in the > >>>>>> AbstractPageableView. > >>>>>> Thus > >>>>>> >> I > >>>>>> >> >> > need > >>>>>> >> >> > to call the database here to figure out the size. > >>>>>> >> >> > > >>>>>> >> >> > Am I doing sonething wrong or have I got to hit the database > >>>>>> twice > >>>>>> for > >>>>>> >> >> > each > >>>>>> >> >> > DataProvider render. > >>>>>> >> >> > > >>>>>> >> >> > Obvously I don't want to hard code a size. Is there any > other > >>>>>> way ? > >>>>>> >> >> > > >>>>>> >> >> > Thanks > >>>>>> >> >> > Wayne > >>>>>> >> >> > > >>>>>> >> >> > > >>>>>> >> >> > >>>>>> >> >> > >>>>>> >> >> ----- > >>>>>> >> >> Michael Sparer > >>>>>> >> >> http://talk-on-tech.blogspot.com > >>>>>> >> >> -- > >>>>>> >> >> View this message in context: > >>>>>> >> >> > >>>>>> >> > >>>>>> > http://www.nabble.com/Is-there-any-other-way--DataProviders-must-hit-the-Db-twice-for-%28possible%29-large-datasets-tp20701684p20702476.html > >>>>>> >> >> Sent from the Wicket - User mailing list archive at > Nabble.com. > >>>>>> >> >> > >>>>>> >> >> > >>>>>> >> >> > >>>>>> > --------------------------------------------------------------------- > >>>>>> >> >> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>>>> >> >> For additional commands, e-mail: [EMAIL PROTECTED] > >>>>>> >> >> > >>>>>> >> >> > >>>>>> >> > > >>>>>> >> > >>>>>> >> > >>>>>> > --------------------------------------------------------------------- > >>>>>> >> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>>>> >> For additional commands, e-mail: [EMAIL PROTECTED] > >>>>>> >> > >>>>>> >> > >>>>>> > > >>>>>> > >>>>>> > --------------------------------------------------------------------- > >>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>>>> For additional commands, e-mail: [EMAIL PROTECTED] > >>>>>> > >>>>>> > >>>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>> For additional commands, e-mail: [EMAIL PROTECTED] > >>>> > >>>> > >>>> > >>> > >>> > >>> ----- > >>> ------- > >>> Stefan Fußenegger > >>> http://talk-on-tech.blogspot.com // looking for a nicer domain ;) > >>> -- > >>> View this message in context: > >>> > http://www.nabble.com/Is-there-any-other-way--DataProviders-must-hit-the-Db-twice-for-%28possible%29-large-datasets-tp20701684p20715382.html > >>> Sent from the Wicket - User mailing list archive at Nabble.com. > >>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>> For additional commands, e-mail: [EMAIL PROTECTED] > >>> > >>> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [EMAIL PROTECTED] > >> For additional commands, e-mail: [EMAIL PROTECTED] > >> > >> > >> > > > > > > ----- > > ------- > > Stefan Fußenegger > > http://talk-on-tech.blogspot.com // looking for a nicer domain ;) > > -- > > View this message in context: > http://www.nabble.com/Is-there-any-other-way--DataProviders-must-hit-the-Db-twice-for-%28possible%29-large-datasets-tp20701684p20723759.html > > Sent from the Wicket - User mailing list archive at Nabble.com. > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
