+dev Good thing to talk about that issue, it's really not an ideal situation to have FileDataModel need double memory at refresh time as it effectively cuts in a half the amount of preferences that can be loaded into per memory per machine.
Only two issues come to my mind if we would do a simple update of the data: a) existing preferences that are not included in the refreshed data will not be removed b) recommenders reading the data at update time see an inconsistent snapshot of the data Let's assume that you only update and add preferences, then a) cannot occur. I'm not sure how much of a problem b) would be in that case, you could miss some cooccurrences when a preference is not added yet and have some similarity value wrong if an update has not yet occurred. I'd guess that is not a big problem as long as you don't cache the inconsistent recommendations resulting from that. So an idea could be to have a FileDataModel that never deletes preferences and have that directly update the data. Any thoughts on this? --sebastian Am 19.11.2010 21:49, schrieb Sean Owen: > ut the old dat
