Geek wins !
I applied Magic Banana's script to the ~2.5MB master file (sans parentheses)
and obtained a ~600KB file of duplicates in ~.2 second. Thank you !
These hostnames came from the combined address ranges of the autonomous
systems 34300,29182,59729,15626,56630,44493, and 48666. I skipped a few CIDR
blocks because those were addressed in previous studies that I did on the
derivatives of example.com. The resulting spreadsheet has ~19,000 rows.
That's as many as all the permutations of example.com and are just as
difficult to resolve with nslookup or dig. It's roughly a fifth of the number
of rows in the master file.
Bear in mind that this is a moving target and would have to be repeated on
almost a daily basis to maintain an accurate list of unresolvable hostnames.
There is apparently no oversight of this issue. Registrars in the Western
World will not let us register a domain name exactly like another one, and
they even deprecate typographic lookalikes. Think of the issues of universal
inoculations, seat belts, inspection of pressure vessels, and driver licenses
...
I am indebted to Magic Banana for his contribution to this effort. He's right
about manual processing; I had even proofread the longer list.
George Langford
