I’ve been thinking about how one would implement an application that only shows
recommendations. This is partly because people want to build such things.
There are many problems with this including cold start and overfit. However
these problems also face MABs and are solved with sampling schemes. So imagine
that you have several models from which to draw recommendations: 1) CF based
recommender, 2) random recommendations, 3) popular recs (by some measure). If
we look at each individual as facing an MAB with a sampling algo trained by
them to pull recs from the 3 (or more) arms. This implies an MAB per user.
The very first visit to the application would randomly draw from the choices
and since there is no user data the recs engine would have to be able to
respond (perhaps with random recs) the same would have to be true of the
popular model (returning random), and random is always happy. The problem with
this is that none of the arms are completely independent and the model driving
each arm will change over time.
The first time a user visits will result in a new MAB for them and will
randomly draw from all arms but may get better responses from popular (with no
user specific data yet in the system for cf). So the sampling will start to
favor popular but will still explore other methods. When enough data is
accumulated to start making good recs, the recommender will start to outperform
popular and will get more of the user’s reinforcement.
This seems to work with several unanswered questions and one problem to
avoid—overfit. We would need a sampling method that would never fully converge
or the user would never get a chance to show their expanding/changing
preferences. The cf recommender will also overfit if non-cf items are not mixed
in. Of the sampling methods I’ve seen for MABs, Greedy will not work but even
with some form of Bayesian/Thompson sampling the question is how to
parameterize the sampling. With too little convergence we get sub-optimal
exploit but we get the same with too much convergence and this will also
overfit the cf recs.
I imagine we could train a meta-model on the mature explore amount by trying
different parameterization and finding if there is one answer for all or we
could resort to heuristic rules—even business rules.
If anyone has read this far, any ideas or comments?