While all this coding was going on, I discovered a flaw in my logic, whereby
my method of
separating not-looked-up IPv4 addresses from the Recent Visitor data was
extracting some
lookalike IPv4 data from the hostnames, including impossible addresses. I
used comm to
select only those addresses that were in both the calculated list and the
Recent Visitor
data, reducing the list of extracted two-octet prefixes from over 50,000 to
around 27,000.
Naturally, now that Magic Banana tells us that awk won't do what I expect,
sure enough, it
stopped causing those user-friendly intermediate writes.
The man sync and info sync pages are not helping me; as yet I haven't the
slightest clue
where sync stores the data in persistent storage; if I knew, I could watch it
develop.
There are now 38 files to be created with long-running nmap scripts; if I
could write one
sync script before I start and not stop it until the last of those 38 files
finishes, and
still be able to watch the occasional disk writes so I could evaluate
progress of the scripts,
that would be ideal. If I could list the 38 output files in advance, that
would save a lot
of scrambling while the nmap searches are onging about ten at a time.
The good news is that when nmap started complaining about 942.103.*.*
addresses, I used
grep to search all the Recent visitor files and found (in a few seconds)
three instances of:
as45942.103.28.157.43.lucknow.sikkanet.com
Dig -x reveals that the actual IP address is 103.28.157.43, not
942.103.28.157, neither of
which ought to contribute to the two-octet prefix list (but 103.28 got in
there anyway !)
If I create a subfolder called Sync under the May2020/IPv4 folder and store a
list of the
expected output filenames to which sync is to be applied in a text file
there, can sync be
made to activate and save cached data to the appropriate filename as soon as
the nmap script
is started ?
George Langford
