On 24/08/18 22:54, Wietse Venema wrote:
> Viktor Dukhovni:
>> Yes, but that'd have to be done by the dictionary lookup layer,
>> possibly in proxymap, based on a suitable signal from the lookup
>> client, but the low-level API (dict_get()) does not presently
>> support any per-lookup flags. So we'd need dict_get_ex() that
>> takes a new utf8 flag and supporting changes throughout the
>> code. This is a major change.
>>
>> Perhaps there's a clever way to avoid this, but I am not seeing it yet.
> I don't think that it is a good idea to signal the query string
> encoding through the Postfix dict(3) API. Primarily because there
> is no legitimate use for non-UTF8 in envelopes and headers, so why
> bother adding complexity now for a dying cause?
>
> Relevant for the $Subject: in UTF8 mode, the Postfix dict API will
> never look up a non-UTF-8 string, because it cannot exist, and it
> will never return a non-UTF-8 lookup result, for the same reasons.
> If non-UTF8 content were allowed in UTF-8 mode, then every string
> would be a potential bomb ready to explode.
>
> Even more relevant for the $Subject: several Postfix lookup tables
> that already block non-UTF8 queries when smtputf8 mode is turned
> off (dict_ldap.c, dict_sqlite.c):
>
> /*
> * Don't frustrate future attempts to make Postfix UTF-8 transparent.
> */
> if ((dict->flags & DICT_FLAG_UTF8_ACTIVE) == 0
> && !valid_utf8_string(name, strlen(name))) {
> if (msg_verbose)
> msg_info("%s: %s: Skipping lookup of non-UTF-8 key '%s'",
> myname, dict_ldap->parser->name, name);
> return (0);
> }
>
> This code has been there for four years already. Never heard a peep.
>
> I think that we should by default extend this to all lookup tables
> and do away with that LATIN1 hack. That would also address the
> problem that underlies the original request. No need for signaling
> the request encoding!
>
> Wietse
Wietse
is the above check done in the single dict modules redundant?
If DICT_FLAG_UTF8_ACTIVE is set it means that the lookup method has
already been overwritten by dict_utf8_lookup
(see util/dict_utf8.c)
/*
* Interpose on the lookup/update/delete methods. It is a conscious
* decision not to tinker with the iterator or destructor.
*/
backup->lookup = dict->lookup;
backup->update = dict->update;
backup->delete = dict->delete;
dict->lookup = dict_utf8_lookup;
dict->update = dict_utf8_update;
dict->delete = dict_utf8_delete;
and where dict_utf8_dict does this check
/*
* Validate and optionally fold the key, and if invalid skip the
request.
*/
if ((fold_res = dict_utf8_check_fold(dict, key, &utf8_err)) == 0) {
msg_warn("%s:%s: non-UTF-8 key \"%s\": %s",
dict->type, dict->name, key, utf8_err);
dict->error = DICT_ERR_NONE;
return (0);
}
John
