I don't think it's the library either. 80K rows load very fast for us. We did experience slow writing back to the database. But after we disable JDBC auto-commit, then INSERT got very fast again.
On Mon, Aug 15, 2011 at 3:50 PM, Sean Owen <[email protected]> wrote: > Something's very wrong there. 80K rows is tiny, and loads in a second > or so from a file. I think you want to figure out where the slow-down > is with some debugging, since I do not think it's the library. Is > something locking a table, excluding reads, for instance? > > On Mon, Aug 15, 2011 at 8:02 PM, Salil Apte <[email protected]> wrote: >> I have been using ReloadFromJDBCDataModel in my project and it has >> been working out well for us. However, there is one problem: reloading >> data with ReloadFromJDBCDataModel tends to be slow. We have about 80k >> ratings in our DB (not a terribly large data set compared to others I >> reckon) and a call to refresh() can take ~10 minutes. Is this to be >> expected? >> >> The slow performance is a bummer for us because our new users have the >> opportunity to rate a few things on our site upon sign-up right before >> they start asking for their first recommendations. But with such a >> long reload time, the users don't get to leverage the ratings they've >> made during the sign-up for their first recommendations (the time it >> takes a user to go from sign-up to first recommendation is < 30 >> seconds). >> >> Any tips or tricks to speed this process up? Can we somehow >> selectively reload data for a user? >> >> -Salil >> >
