I believe that OP was asking about the local gateway/router, not the ISP's
servers.
It's a tricky issue for sure. I've tried to minimise the amount of non-free
firmware/software I use in this regard by eliminating the router all
together, and using a "dumb" ADSL2+ bridge modem connected directly to my
server, which runs free software (Debian actually). My Debian server acts as
a router and also has a freedom-compatible WiFi PCIe card in it that allows
it to act as a WiFi base station. It also has 2 NICs, so one is connected to
the modem in PPPoE mode, and the other is connected to a layer-2 switch.
There's still the issue of the non-free firmware in the modem, which is a
problem. However I don't know of a solution to this. There's also no free
bios currently available for any of my hardware.