Dropping 10.5 seems like a good choice if it results in faster development, makes the project more attractive to newcomers and enables use of new technologies (the last part is a given really).
As far as users are concerned, 10.4 has been dropped for a while so best to leave that out of the question of statistics really, as what Adium 1.5 supports doesn't affect them. You'd basically be letting go of around 21% users if you were to release 1.5 at this very second, and it's best to assume a few % of these 10.5 users aren't even running the latest version of Adium (Sparkle doesn't filter based on what version of Adium they are running), and then assume a few more % of these users will upgrade their machines/OS with the release of Lion. So you're looking more at 10-15% of users that'll be left out, around the time Adium 1.5/Lion are released, which would eventually drop to single digits. I think the most important question to ask though is how long (if at all) would the 1.4 branch be maintained for connection breakages for 10.5 users?