Hi, what can happen is that the passphrase got saved in the keyring provided by your desktop environment, for example the GNOME Keyring. This gets unlocked when you enter your password on logon. The program to manage the gnome keyring is called Seahorse, for KDE there's the KDE Wallet Manager. If that's the case, and you don't want that, you probably need to setup another pinentry program. But I think some instances of the session keyrings can emulate the gpg agent by themselves.
Good luck, Matthias P.S.: Sorry if anyone got this message twice, my list setup seems to be broken and I used the wrong email address.
