Hi Way Cool, How many users and items do you have, how many similar items per item do you store?
And what's your scenario? Being limited to 4GB in a production machine seems a little odd. --sebastian On 21.06.2012 23:16, Way Cool wrote: > Hi, guys, > > For item-based recommendation, I pre-calculated the item similarities on > Hadoop per algorithm, which generated 20m rows each. The problem now is I > can't just load them into memory via MySQLJDBCInMemoryItemSimilarity with > 4GB memory. I tried MySQLJDBCItemSimilarity, however it's way too slow. > What are the alternatives? > > For user-based recommendation, I can't load 100m lines of data model from > FileDataModel into memory. It ran out of memory after 20m lines. The same > issue with JDBCDataModel is way too slow. Does anyone precalculate the user > similarities before and recommend items to a user? > > Anyone had the similar issues before? > > Thanks, > > YG >
