fkoyer opened a new pull request, #48:
URL: https://github.com/apache/spamassassin/pull/48
## Identify MIME part content by magic bytes in effective_type()
`effective_type()` now sniffs the leading bytes of a part whose declared type
(or filename extension) is a generic or non-text label, letting the real
format win — e.g. a PDF mislabelled `application/octet-stream` is recognised
as
`application/pdf`. Adds a `Util::content_type_from_magic()` helper and
extends
the extension/MIME maps. New test `t/node_sniff.t`; `t/extracttext.t` updated
for the new behavior.
## Behavior change: MIME part bodies are kept in memory, not spooled to disk
This PR removes the temp-file spool in `_parse_normal()` outright (no
size threshold — all part bodies now stay in memory). Previously every part
that was not `text/plain`, `text/html` or `message/*` was written to a
temporary file.
Rationale — both premises behind the spool have stopped holding:
- **It did not actually bound memory.** A spooled part is read back *in full*
by both `raw()` and `decode()` (neither seeks — both slurp the whole file),
so any part something looks at ends up resident anyway, plus a write and a
read-back. And the part's bytes are already held in `{pristine_body}` and
`{pristine_msg}` for the lifetime of the scan — so the spool only ever
duplicated bytes that were resident regardless; it never capped peak RSS.
- **It targeted the wrong parts.** The type test predates the MIME-part
handlers. PDFs, images, archives and ICS invites are all opened now, so the
spool had come to write out exactly the parts most likely to be read, while
keeping an arbitrarily large `text/html` part in memory.
Measured on a 5.4 MB message with a 4 MB attachment: **0.2 MB saved, ~20%
slower** once anything read the part back.
It also removes an ordering trap for the sniffing below: `effective_type()`
is
called before `{raw}` is assigned, so dropping the spool lets the magic-byte
check run at the right point.
Tested: `make test` passes locally (full suite).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
