Nobel laureate, admittedly only for Economics, and also Turing Award winner, Herb Simon did a lot of research on the attention economy.
The people who should have paid attention, us consumers, didn't - and the people who did pay attention to his research were the marketers, who did pay a lot of attention. Simply put, attention span is limited, and we ought to spend it as carefully as we spend money. Yet, today when you switch on the TV or enter a store, your attention is stolen from you. So too with your smart phone - email notifications, game notifications, IM notifications and what not you are paying more attention outside than inside. No surprise then that the mega store knows more about you than you for the vast majority who pay no attention to introspection. Speaking from experience, introspection of the self is one of the most private, and luxurious acts one can do. It is after all paying attention to yourself.