Wietse, Viktor
It seems that this is the problematic part of the pcre pattern
>
/^Content-(Disposition|Type).*name\s*=\s*("(?:[^"]|\\")*|[^();:,\/<>\@\"?=<>\[\]\
]*)
I tested by replacing star by star with explicit limits and found that
limits avoid the segfault. I replaced the last two stars with limits up
to 10000 chars
>
/^Content-(Disposition|Type).*name\s*=\s*("(?:[^"]|\\"){0,10000}|[^();:,\/<>\@\"?=<>\[\]\
]{0,10000})
no segfault but I think the whole header has not been processed as its
longer than 10'000 chars. Tried with higher values than 10000 but then
postmap issues a warning that the number is too big.
> postmap: warning: pcre map /etc/postfix/mime_header_checks.pcre, line
21: error in regex at offset 63: number too big in {} quantifier
After solving the "too big number" issue postmap said that the pcre
pattern is too big :-)
> postmap: warning: pcre map /etc/postfix/mime_header_checks.pcre, line
21: error in regex at offset 352: regular expression is too large
So the problem is definitely our pattern in combination with a VERY big
mime part header. We're on introducing limits where we can in our patterns.
Anyway I think that this should not end in such an ugly error where
postfix cleanup goes south because of such header/pattern combination.
--
Cheers
tobi
Am 14.05.20 um 09:13 schrieb Viktor Dukhovni:
> On Thu, May 14, 2020 at 08:53:42AM +0200, Tobi wrote:
>
>> using your suggestion with valgrind I found that the main-stacksize
>> seems to be the problem. By using --main-stacksize with different values
>> I found that the last working value is 59449345, changing to ...344 let
>> postmap segfault:
>
> But whatever the stacksize, the PCRE library must not segfault. We need
> to be sure that it returns an error to the caller, rather than blows up.
>
>>> ==7988== Stack overflow in thread #1: can't grow stack to 0x1ffb74f000
>>
>> We will try allowing more stacksize by /etc/security/limits.conf to
>> postfix and re-design our pcre pattern(s) to avoid (where possible) the
>> usage of .* without any limits :-)
>
> Keeping your (ir)regular expressions simpler is always a good idea.
> Especially avoid costly use of multiple '.*' patterns in the same
> expression. Not a good idea:
>
> /a.*b.*c.*d.*...z/
>
> Perhaps there's something we can do with default resource limits.
> From the pcre2pattern(2) manpage:
>
> Setting match resource limits
>
> The pcre2_match() function contains a counter that is incremented
> every time it goes round its main loop. The caller of
> pcre2_match() can set a limit on this counter, which therefore
> limits the amount of computing resource used for a match. The
> maximum depth of nested backtracking can also be limited; this
> indirectly restricts the amount of heap memory that is used, but
> there is also an explicit memory limit that can be set.
>
> These facilities are provided to catch runaway matches that are
> provoked by patterns with huge matching trees. A common example
> is a pattern with nested unlimited repeats applied to a long
> string that does not match. When one of these limits is reached,
> pcre2_match() gives an error return. The limits can also be set
> by items at the start of the pattern of the form
>
> (*LIMIT_HEAP=d)
> (*LIMIT_MATCH=d)
> (*LIMIT_DEPTH=d)
>
> where d is any number of decimal digits. However, the value of
> the setting must be less than the value set (or defaulted) by the
> caller of pcre2_match() for it to have any effect. In other
> words, the pattern writer can lower the limits set by the
> programmer, but not raise them. If there is more than one setting
> of one of these limits, the lower value is used. The heap limit
> is specified in kibibytes (units of 1024 bytes).
>
> Prior to release 10.30, LIMIT_DEPTH was called LIMIT_RECURSION.
> This name is still recognized for backwards compatibility.
>
> The heap limit applies only when the pcre2_match() or
> pcre2_dfa_match() interpreters are used for matching. It does not
> apply to JIT. The match limit is used (but in a different way)
> when JIT is being used, or when pcre2_dfa_match() is called, to
> limit computing resource usage by those matching functions. The
> depth limit is ignored by JIT but is relevant for DFA matching,
> which uses function recursion for recursions within the
> pattern and for lookaround assertions and atomic groups. In this
> case, the depth limit controls the depth of such recursion.
>
