Just closing the loop on this...
So, nobody had any advice as to how to troubleshoot this and I stopped looking for root cause and started looking for a fix of any sort. Turns out that was to make sure nothing was using /gnu/store and then `rm -rf /gnu/* /var/guix` and re-install guix. All of the nodes which were exhibiting this behavior stopped doing so after a re-install. My only theory is that there is something I am doing when managing the guix binary install that will occasionally result in the systems getting into this state. Perhaps something to do with the TLS libs.
