is lowering timeoutremote dangerous

[Fox]

My timeoutremote was the default of 20 minutes waiting for a response 
from the mail server.  This seems excessive to me.  I would like to 
lower it to between 1 to 5 minutes so I don't have a bunch of 
qmail-remote processes loitering doing nothing.  

Is there any reason the default is so high at 20 minutes?  Won't most 
mail servers respond within 15 seconds or less?  Will setting it lower 
adversely impact my mail delivery capability?

Thanks,
Fox