You do, indeed, need to configure PAM for IMAP use. Most likely, what you are missing is an /etc/pam.d/imap file but I am not an expert on PAM.
Does this help?
-- Mark --
http://staff.washington.edu/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum.
