Hi,
I've got a question regarding how to split data (e.g. MovieLens) into training and testing data when I want to test the performance of CF-based recommender. In particular, I want to focus on the metrics including RMSE, precision and recall (for precision and recall, we convert any ratings higher than 3 to LIKE and anything else DISLIKE). If for each user, we randomly split his data by a ration of 8:2 (80% for training and 20% for testing), then we may end up with scenario where some of the items (e.g. movies) in the test data fail to appear in the training data. Due to the cold-start item issue, the CF-based recommender will not be able to predict a rating for such items. However, this is not issue for content-based recommender which is able to predict a rating for any items. I was wondering how people usually go about this issue when they want to compare the performance of a CF-based recommender and a content-based recommender on the metrics such as RMSE, precision and recall. Do they simply eliminate these items (in test data, but not in training data) from evaluation on CF-based recommender or do they have to make sure that each item appear in both training and test data so that CF can make prediction on every item in the test data? Thanks, James
