apt in jessie does not have the same problem. Have there been changes in that area?
OTOH, is is also possible the perceived slow-down is due to apt-file now using apt's transport mechanisms (and apt-file's pdiffs being a lot larger than apt-get's). I could check that tonight if you think it would be useful.
