Hello, I propose to add a new tls_key_log directive to record TLS pre-master secret (and related encryption details) for to- and from-Squid TLS connections. This very useful triage feature is common for browsers and some networking tools. Wireshark supports it[1]. You might know it as SSLKEYLOGFILE. It has been requested by several Squid admins. A draft documentation of the proposed directive is at the end of this email.
[1] https://wiki.wireshark.org/TLS#Using_the_.28Pre.29-Master-Secret If you have any feature scope adjustments, implementation wishes, or objections to this feature going in, please let me know! FTR, we have considered providing similar support by adding new logformat %code(s) to the existing access_log. While doing so would reduce and simplify code changes, we ultimately rejected that design because the combination of the following factors renders it inferior and insufficient on several levels: 1. Some TLS connections are reflected in access logs a long time after they are established (e.g., when the connection serves a long CONNECT tunnel). During triage, admins may need the ability to decipher a live connection much sooner. 2. Some TLS connections are never reflected in access logs at all (e.g., when Squid opens but does not use an outgoing TLS connection or crashes when parsing the first request on an incoming one). Info gaps often create triage suspicions: Did we drop something else? 3. A single access_log record may correspond to many from-Squid connections, especially when Squid retries peer failures. Logging keys for all these connections would require accumulating the keys in the master transaction and then dumping them as a part of a new %code. Adding (and dumping) repeated ALE fields is awkward. 4. Manually creating ACLs to limit access log records to the first transaction on a connection would be a must for most deployments using this feature. Doing so is far from trivial and related configuration errors are difficult to triage. We could add a new ACL type for this purpose, but even that is tricky because a single master transaction may have many associated connections. And logging secrets for every transaction creates too much noise. 5. Configuration flexibility offered by logformat is likely to remain largely unused by the new feature because tools like Wireshark _automatically_ find relevant records when deciphering captured traffic. Augmenting these logs with other transaction details (typical for access log uses) would be mostly useless. 6. New %codes would be awkward to use in a regular access log because they may expand into a variable number of lines, going against the traditional line-oriented, "fixed" format access log use. While some of the above items have workarounds, a few do not, and the whole combination looks rather grim/unfriendly. We should attack this problem from the other end -- a new simple configuration dedicated to this useful feature. We propose to structure this new directive so that it is easy to add advanced access_log-like features later if needed (while reusing the corresponding access_log code). For example, if users find that they want to maintain multiple TLS key logs or augment log records with connection details, we can add that support by borrowing access_log options and code without backward compatibility concerns. The new required "if" keyword in front of the ACL list allows for seamless addition of new directive options in the future. Cheers, Alex. ---------- draft squid.conf directive documentation ------------ tls_key_log Configures whether and where Squid records pre-master secret and related encryption details for TLS connections accepted or established by Squid. These connections include connections accepted at https_port, TLS connections opened to origin servers/cache_peers/ICAP services, and TLS tunnels bumped by Squid using the SslBump feature. This log (a.k.a. SSLKEYLOGFILE) is meant for triage with traffic inspection tools like Wireshark. tls_key_log <filename> if <acl>... WARNING: This log allows anybody to decrypt the corresponding encrypted TLS connections, both in-flight and postmortem. At most one log file is supported at this time. Repeated tls_key_log directives are treated as fatal configuration errors. By default, no log is created or updated. If the log file does not exist, Squid creates it. Otherwise, Squid appends an existing log file. The directive is consulted whenever a TLS connection is accepted or established by Squid. TLS connections that fail the handshake may be logged if Squid got enough information to form a log record. A record is logged only if all of the configured ACLs match. Squid does not buffer these log records -- the worker blocks until each record is written. File system buffering may speed things up, but consider placing this triage log in a memory-based partition. This log is rotated based on the logfile_rotate settings. Logging errors are reported to cache.log but are otherwise ignored. While transport-related ACLs like src and dst should work, Squid may not have access to higher-level information. For example, when logging accepted https_port connections, Squid does not yet have access to the expected HTTPS request. Similarly, an HTTPS response is not available when logging most TLS connections established by Squid. The log record format is meant to be compatible with TLS deciphering features of Wireshark which include support for TLS v1.3 fields such as CLIENT_EARLY_TRAFFIC_SECRET and SERVER_TRAFFIC_SECRET_0. A single log record usually spans multiple lines. Technical documentation for that format is maintained inside the Wireshark code (e.g., see tls_keylog_process_lines() comments as of Wireshark commit e3d44136f0f0026c5e893fa249f458073f3b7328). This clause only supports fast acl types. See http://wiki.squid-cache.org/SquidFaq/SquidAcl for details. _______________________________________________ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev