Hello,
The attached patch modifies Squid to run /tmp/squid-on-fatal.sh on
fatal errors. A sample script is also attached. I needed this trick to
triage a complicated bug related to Squid shared memory usage.In general, running such a script helps record the environment state _before_ Squid exits, which is important when the problem is related to Squid resource usage that often becomes invisible after the Squid process is gone: connections, FDs, shared memory segments, etc. The patch needs more work to make this behavior and the script name squid.conf-configurable. We should also pass Squid PID and Squid process "role" to the script as parameters so that the script does not have to guess them. Do you want this feature in Squid? If yes, which squid.conf directive name do you think works the best? Here are some options to consider: on_crash ... on_fatal_exit .. on_bad_exit ... on_unexpected_exit ... on_exit <success|failure> ... The directive parameter could be program=... handler=... which would allow us to easily add more exit-handling options later if needed (e.g., free_all_memory, keep_shared_memory, etc.). Please note that Squid already does something similar on startup. See squid_start_script in main.cc. That hack should be made configurable too, of course (on_start, on_launch, etc.). Thank you, Alex.
Modified Squid to run /tmp/squid-on-fatal.sh on fatal errors.
This helps record the environment state _before_ Squid exits,
which is important when the problem is related to Squid
resource usage: connections, FDs, shared memory segments, etc.
Needs more work to make this behavior and the script name
squid.conf-configurable. We should also pass Squid PID to
the script as a parameter so that it does not have to guess.
=== modified file 'src/fatal.cc'
--- src/fatal.cc 2015-01-13 07:25:36 +0000
+++ src/fatal.cc 2015-12-01 01:21:16 +0000
@@ -1,55 +1,87 @@
/*
* Copyright (C) 1996-2015 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#include "squid.h"
#include "Debug.h"
#include "fatal.h"
#include "globals.h"
#include "SwapDir.h"
#include "tools.h"
+#if HAVE_SYS_STAT_H
+#include <sys/stat.h>
+#endif
+#if HAVE_UNISTD_H
+#include <unistd.h>
+#endif
+
+/// Final reporting/non-state-changing parts of fatal().
+/// Not static to simplify temporary hacks that report Squid state
+/// [while bypassing nearly-fatal events].
+void
+OnFatal()
+{
+ PrintRusage();
+ dumpMallocStats();
+
+ // Run an on-fatal script if any; TODO: Remove or make configurable.
+ const char *script = "/tmp/squid-on-fatal.sh";
+ struct stat st;
+ if (stat(script, &st) == 0) {
+ debugs(54, 2, "running " << script);
+ if (system(script) != 0) {
+ const int savedError = errno;
+ debugs(54, DBG_IMPORTANT, script << " failure: " << xstrerr(savedError));
+ }
+ debugs(54, 5, "done running " << script);
+ } else {
+ const int savedError = errno;
+ debugs(54, 2, "assuming no " << script << ": " << xstrerr(savedError));
+ }
+
+ fflush(debug_log);
+}
+
static void
fatal_common(const char *message)
{
#if HAVE_SYSLOG
syslog(LOG_ALERT, "%s", message);
#endif
fprintf(debug_log, "FATAL: %s\n", message);
if (Debug::log_stderr > 0 && debug_log != stderr)
fprintf(stderr, "FATAL: %s\n", message);
fprintf(debug_log, "Squid Cache (Version %s): Terminated abnormally.\n",
version_string);
fflush(debug_log);
- PrintRusage();
-
- dumpMallocStats();
+ OnFatal();
}
void
fatal(const char *message)
{
/* suppress secondary errors from the dying */
shutting_down = 1;
releaseServerSockets();
/* check for store_dirs_rebuilding because fatal() is often
* used in early initialization phases, long before we ever
* get to the store log. */
/* XXX: this should be turned into a callback-on-fatal, or
* a mandatory-shutdown-event or something like that.
* - RBC 20060819
*/
/*
* DPW 2007-07-06
squid-on-fatal.sh
Description: Bourne shell script
_______________________________________________ squid-dev mailing list [email protected] http://lists.squid-cache.org/listinfo/squid-dev
