I've only recently been made aware of "ssh-add -c" and tried it out: For me it does work at add-time, then fails reporting agent failure.
My best guess is that with gnome-keyring and ssh-agent both having no DISPLAY= variable set, it's unable to ask for confirmation. I'll file a separate bug, but this one can be closed?
