Is there a way to selectively reload data from the database for a user? That way, we wouldn't have to pull down 80k records on every reload?
On Mon, Aug 15, 2011 at 1:59 PM, Sean Owen <[email protected]> wrote: > That's more reasonable. It sounds a bit long still but could believe > it is due to the overhead of reading everything from the database. > > It seems very expensive to reload everything on every user change -- > that's not quite what it was meant for. Though I imagine you could get > away with it for small data sets. > > You can hack it up a little to actually write into the in-memory > representation at run-time. That would be a lot better. Look at the > getRawData() method (something like that -- it's not in front of me). > It's not such a sin to edit that directly if you want to go this way. > > On Mon, Aug 15, 2011 at 9:29 PM, Salil Apte <[email protected]> wrote: >> Apologies, I typed that email without having had my coffee. I meant it >> takes 10 seconds to reload. >> >> On Mon, Aug 15, 2011 at 1:16 PM, Daniel Xiaodan Zhou >> <[email protected]> wrote: >>> I don't think it's the library either. 80K rows load very fast for us. >>> We did experience slow writing back to the database. But after we >>> disable JDBC auto-commit, then INSERT got very fast again. >>> >>> On Mon, Aug 15, 2011 at 3:50 PM, Sean Owen <[email protected]> wrote: >>>> Something's very wrong there. 80K rows is tiny, and loads in a second >>>> or so from a file. I think you want to figure out where the slow-down >>>> is with some debugging, since I do not think it's the library. Is >>>> something locking a table, excluding reads, for instance? >>>> >>>> On Mon, Aug 15, 2011 at 8:02 PM, Salil Apte <[email protected]> wrote: >>>>> I have been using ReloadFromJDBCDataModel in my project and it has >>>>> been working out well for us. However, there is one problem: reloading >>>>> data with ReloadFromJDBCDataModel tends to be slow. We have about 80k >>>>> ratings in our DB (not a terribly large data set compared to others I >>>>> reckon) and a call to refresh() can take ~10 minutes. Is this to be >>>>> expected? >>>>> >>>>> The slow performance is a bummer for us because our new users have the >>>>> opportunity to rate a few things on our site upon sign-up right before >>>>> they start asking for their first recommendations. But with such a >>>>> long reload time, the users don't get to leverage the ratings they've >>>>> made during the sign-up for their first recommendations (the time it >>>>> takes a user to go from sign-up to first recommendation is < 30 >>>>> seconds). >>>>> >>>>> Any tips or tricks to speed this process up? Can we somehow >>>>> selectively reload data for a user? >>>>> >>>>> -Salil >>>>> >>>> >>> >> >
