I don't think Apache::compat is a culprit, it just doesn't mess with the port methods, however I won't single out that possibility completely,
the behavior of ap_get_server_port looks to be very different between apache 1.3 and 2.0, so I suspect the difference is really there.
I'd try playing with different UseCanonicalName directives and see what happens.
--Geoff