My dig scripts had been curtailed by a 'dig -' error caused because the twixt
file was getting chopped off mid-output, ending
I have been explaining you the problem (and its solution) many times now: the
commands on both side of a pipe, |, run in parallel: you cannot have command1
write a regular file and reliably have, somewhere else in the command line,
command2 read it. The file may not even exist yet when command2 tries to
read it. Use pipes, not regular files.
I divided my ~30,000 row input file into ten ~3000 row files, and now the
first three of those completed with no errors, but only about half-way
through the subdivided input files
That is no reliable solution. Obviously. Trying to fix things without
understanding the problem does not lead anywhere. And to understand the
problem, you need to understand how pipes work.
It appears to me that the awk script runs first, and in a very short time,
followed one-by-one by the dig and grep scripts.
For the n-th time: they run in parallel. Not in a sequence.
Right now, the following syntax looks better and processes the entirety of
the subdivided input file
Again: do not subdivide the input file.
Also, I do not understand how you can deem the syntax of the referred command
line "better" than giving a name (e.g., "find-IPv4-addr.awk") to the AWK
script I gave you in
https://trisquel.info/forum/how-can-one-capture-failed-resolve-output-console-during-nmap-scan#comment-141556
, giving another name (e.g., "grep-URLs-after-dig.awk") for the two-line
script in
https://trisquel.info/forum/how-can-one-capture-failed-resolve-output-console-during-nmap-scan#comment-141639
and writing (after turning both scripts executable):
$ ./find-IPv4-addr.awk hostname_list | dig -f - | ./grep-URLs-after-dig.awk
