I doubt that query compilation is the major cost here. The problem is that too many records are being moved too often.
Sent from my iPad On Aug 15, 2011, at 10:23 PM, Lance Norskog <[email protected]> wrote: > The standard advice also applies: use stored procedures if you can. If > not, use 'prepared statements' because they precompile the database > execution plan. Also, different DBs have their quirks in re batching > reads and returning them. > > On Mon, Aug 15, 2011 at 1:59 PM, Sean Owen <[email protected]> wrote: >> That's more reasonable. It sounds a bit long still but could believe >> it is due to the overhead of reading everything from the database. >> >> It seems very expensive to reload everything on every user change -- >> that's not quite what it was meant for. Though I imagine you could get >> away with it for small data sets. >> >> You can hack it up a little to actually write into the in-memory >> representation at run-time. That would be a lot better. Look at the >> getRawData() method (something like that -- it's not in front of me). >> It's not such a sin to edit that directly if you want to go this way. >> >> On Mon, Aug 15, 2011 at 9:29 PM, Salil Apte <[email protected]> wrote: >>> Apologies, I typed that email without having had my coffee. I meant it >>> takes 10 seconds to reload. >>> >>> On Mon, Aug 15, 2011 at 1:16 PM, Daniel Xiaodan Zhou >>> <[email protected]> wrote: >>>> I don't think it's the library either. 80K rows load very fast for us. >>>> We did experience slow writing back to the database. But after we >>>> disable JDBC auto-commit, then INSERT got very fast again. >>>> >>>> On Mon, Aug 15, 2011 at 3:50 PM, Sean Owen <[email protected]> wrote: >>>>> Something's very wrong there. 80K rows is tiny, and loads in a second >>>>> or so from a file. I think you want to figure out where the slow-down >>>>> is with some debugging, since I do not think it's the library. Is >>>>> something locking a table, excluding reads, for instance? >>>>> >>>>> On Mon, Aug 15, 2011 at 8:02 PM, Salil Apte <[email protected]> wrote: >>>>>> I have been using ReloadFromJDBCDataModel in my project and it has >>>>>> been working out well for us. However, there is one problem: reloading >>>>>> data with ReloadFromJDBCDataModel tends to be slow. We have about 80k >>>>>> ratings in our DB (not a terribly large data set compared to others I >>>>>> reckon) and a call to refresh() can take ~10 minutes. Is this to be >>>>>> expected? >>>>>> >>>>>> The slow performance is a bummer for us because our new users have the >>>>>> opportunity to rate a few things on our site upon sign-up right before >>>>>> they start asking for their first recommendations. But with such a >>>>>> long reload time, the users don't get to leverage the ratings they've >>>>>> made during the sign-up for their first recommendations (the time it >>>>>> takes a user to go from sign-up to first recommendation is < 30 >>>>>> seconds). >>>>>> >>>>>> Any tips or tricks to speed this process up? Can we somehow >>>>>> selectively reload data for a user? >>>>>> >>>>>> -Salil >>>>>> >>>>> >>>> >>> >> > > > > -- > Lance Norskog > [email protected]
