There has been some progress, regression, and progress again on the git server side of things. On Friday I switched git over to the new server around noon. Initial tests were good. Then about 5pm I saw the first problem reports. Permission denied pushing commits. Drat! I switched git back to the old server while debugging the problem.
The problem was that I had tested before the database migration and the database migration broke the nfs server groups. That has been fixed. I am once again switching git onto the new server. I might as well mention here that the database has been migrated from the old internal to the new internal0 VM. That movement went very well and was a low stress move. I think I will go so far as to say that it was completely invisible to users. If all continues I will ask Ruben to shutdown the old VM on Monday. The detail of the problem Friday was that the rpc.mountd daemon on vcs was started before having changed the libnss-mysql.cfg file and retained the configuration past after having changed the embedded IP address and account credentials. This meant that after having shutdown the old mysqld that it could no longer get user account group information resulting in loss of permission. I should have either restarted all of the long running daemons such as that or have rebooted to ensure that all were restarted after the configuration change. This same problem would also affect download too and I will do this there when the time comes for it. This resulted in git users seeing permission denied errors when trying to push commits. Sorry. Fixed now. Moving forward again. Bob
