So, I have a most troubling issue: Intermittent blocking by our pre-commit hook, with the only output being:
svn: 'pre-commit' hook failed (did not exit cleanly: apr_exit_why_e was 2, exitcode was -1073741502). with no output. This hook is very simple: just checks that the log for the commit contains something (anything), and succeed if there is content in the log. Otherwise fail. Straightforward, nothing special, log requirement on commits. This error only pops up intermittently: a service restart of Apache will give us about 24-48 hours without any errors. A server restart usually buys 1-2 weeks. Either way, this error comes back, and when it happens, there's usually about a 50-50 chance that when the user tries a second time, they do not get this error and can commit successfully, having changed nothing on their end or on the server. When the problem doesn't somehow fix itself, it usually persists for a couple of hours until we restart something server-side. But here's the kicker: The svn binaries that are checking the log for content are separate from the binaries used by the Apache service (Why is a good question. The answer is: I have no clue). Switching which binaries are used doesn't help at all, and makes it strange that an Apache restart alleviates things. I also have found no correlations to the amount of traffic we're receiving to the server, the amount of disk space used, disk i/o, or anything other machine metric. Memory is around 30% used, processor averages 50% load. I've tried troubleshooting the running httpd process and its spawns, found noerrors or unusual activity, and this blocking doesn't seem to affect all repositories, just a random few at a time. It's also extremely hard to reproduce intentionally. This server has ran for years with no issues, and nothing in our environment has changed recently. It also affects both Windows and Linux clients, both in and out of the U.S., and locale settings are universal. I've run stuck after pursuing this for the last 2 months with no success. If anyone could offer some more troubleshooting tips or possibilities, that would be a wonderful blessing. Some details on the server and SVN: Server: Windows 2008 R2 (VM) running Apache 2.2.24 (64bit) Subversion: 1.7.14 and 1.6.2 (Server has1.7.14) Traffic: Servers 7k+ repos (2k actively used), ~26mil hits a day (mostly ALM tools like Jenkins) Disk: 2.5 TB NTFS format
