On 02/09/18 13:17, Eric Wong wrote:
> In addition to the git object_id (blob SHA-1) and Message-Id
> header; it seems necessary to introduce an in-between identifier
> for deduplicating which isn't as loose as Message-Id or as
> strict as object_id: content_id
> I think a hash of the following raw headers + raw body will
> suffice:
>       Subject, From, Date, Message-Id, References, To, Cc,
>       In-Reply-To, MIME-Version, Content-Type,
>       Content-Disposition, Content-Transfer-Encoding

That's similar to what ARC/DKIM do. E.g. in my mailbox the message has
the following headers sealed:


If we trim the arc- and list-specific headers, that's:


I'm not sure we should care about content-transfer-encoding, because
that can be mangled by intermediate MTAs (at least that used to happen
all the time in the past -- not sure if it's still the case).

> List-Id, X-Mailing-List should be left out so different
> readers/lists can share spam removals in cross posts.

Note, that mailing lists that modify the Subject header (e.g. to add
[mailinglist] identifier) will also be impacted similarly.

> (*) I noticed the first Received: header (last hop) is missing
>     from the cregit sources; but the first remaining Received:
>     header also includes the identity of the recipient in more
>     recent mails...

I specifically sanitized all Received: headers that didn't say "by
vger.kernel.org" because these are donated by individual users and I
didn't want to expose their potentially private info.

Konstantin Ryabitsev
Director, IT Infrastructure Security
The Linux Foundation

Attachment: signature.asc
Description: OpenPGP digital signature

Reply via email to