I concur. I have always had a terrible experience using the GNOME network manager, with all sorts of different cards. Thankfully, my current wifi card is well-supported with free firmware, and so I do use the network manager. In the past, I simply connected with wpa_supplicant before I started X, and never touched the network manager. Worked like a charm.
Try just using wpa_supplicant.
