On Sun, Dec 23, 2018 at 12:06:02PM +0100, Aham Brahmasmi wrote:
> Bonjour Monsieur Gilles,
>
> Merci beaucoup for your exhaustive explanations.
>
> > > 1) What is the difference between the "report" and "filter" prefixes?
> > > My current understanding is that "report" is oriented towards reporting
> > > and "filter" is oriented towards writing filters.
> > >
> >
> > very simple:
> >
> > - event reporting lets smtpd notify filters of any event that leads to
> > a change in the SMTP session, they are informative and do not accept
> > any answer.
> >
> > - event filtering lets smtpd notify filters of any event that leads to
> > a decision in the SMTP session, they are decisive and require that a
> > filter provides a decision.
> >
> > report lines are one way from smtpd to filters, filters must not reply
> >
> > filter lines are two way between smtpd and filters, for any request an
> > answer is required.
> >
> > both reporting and filtering are "subscribed" so filters will only get
> > the events they subscribed to.
>
> So in case one wants to reject a session based only on reporting events,
> one still has to wait till the filter events start coming in. Or one
> could write a built-in like fcrdns. Is this correct?
>
To reject a session based only on reporting events, you should subscribe
to the relevant events in order to gather the info you want, then to one
filter event to actually perform the action of rejecting.
For example, I could subscribe to tx-connect, tx-helo and tx-ehlo to get
source address, rdns, fcrdns and helo name from reporting events, but to
reject the session based on this information I must decide when I want a
rejection (do i want to reject at MAIL FROM ? RCPT TO ? DATA ? COMMIT ?)
and register for appropriate filter event to actually reject a session.
Builtin filters are a bit special because we are going to be VERY, VERY,
VERY selective about them. Anything that can be done with a builtin, can
be done with a proc filter and proc filters don't run in the same memory
space so ... a good approach is to write a proc filter and if we find it
makes sense to convert to a builtin, it can be discussed.
> > > 3) Are there time limits for a filter to return response?
> > >
> >
> > time limits are not implemented yet, but yes there will be a time limit,
> > very likely related to the SMTP session timeouts.
> >
> > it's not tricky to implement, it'll be a notification sent to the filter
> > that the last query timed-out and it shouldn't respond anymore.
> >
> > this will result in a Temporary Failure in the SMTP session, filters are
> > not allowed to exceed that timeout.
>
> So if in a chain {f1,f2,f3}, if f2 takes too much time to respond, both
> f1 and f3 will be notified of the session timeout.
>
yes, definitely.
in filter chains, all filters always receive all _reporting_ events.
this is a reason why report and filters are separate, if you look at the
connect phase, all filters will receive the tx-connect report event THEN
the connect filter event is triggered and all filters may not receive it
because first filter may reject. if it rejects, then the reporting event
tx-disconnected is sent to all filters.
the action of a filter will never prevent all filters from receiving the
report events.
> > > 5) Could we have the filter builtin for helo be different from the
> > > ehlo? It might be instructional to understand that the client asked
> > > for ehlo.
> > > { FILTER_EHLO, "ehlo", filter_builtins_helo },
> >
> > it is already the case, helo and ehlo are different filter hooks:
> >
> > filter foo1 builtin helo [...]
> > filter foo1 builtin ehlo [...]
> >
> > the same applies for proc filters, they can subscribe to helo and ehlo
> > as different filtering events.
>
> I may be wrong here, but I was unable to find a filter_builtins_ehlo in
> the code. From lka_filter.c
> ..
> static int filter_builtins_notimpl(struct filter_session *, struct filter
> *, uint64_t, const char *);
> static int filter_builtins_connect(struct filter_session *, struct filter
> *, uint64_t, const char *);
> static int filter_builtins_helo(struct filter_session *, struct filter *,
> uint64_t, const char *);
> static int filter_builtins_mail_from(struct filter_session *, struct
> filter *, uint64_t, const char *);
> static int filter_builtins_rcpt_to(struct filter_session *, struct filter
> *, uint64_t, const char *);
> ..
> { FILTER_CONNECT, "connect", filter_builtins_connect },
> { FILTER_HELO, "helo", filter_builtins_helo },
> { FILTER_EHLO, "ehlo", filter_builtins_helo },
> { FILTER_STARTTLS, "starttls", filter_builtins_notimpl },
> ..
>
yes, the same implementation of the builtin is used so I factored it but
the filter phase and reporting events differentiate them.
don't worry about that I ran a filter rejecting HELO and accepting EHLO,
the two are distinct as far as filtering is concerned.
> > > 7) In lka_filter.c, if a filter feeds back more than LINE_MAX, should
> > > we handle that?
> > >
> > > (void)strlcpy(buffer, line, sizeof buffer);
> > >
> >
> > that's an interesting question.
> >
> > LINE_MAX is not the correct value but we need to have a maximum value
> > for the line and filters will need to ensure they don't produce lines
> > bigger than these.
>
> Understood. I based the LINE_MAX on the following within lka_filter.c:
> int
> lka_filter_process_response(const char *name, const char *line)
> {
> ...
> char buffer[LINE_MAX];
> ...
> (void)strlcpy(buffer, line, sizeof buffer);
>
yes, I had to start with something.
to be very transparent my goal was to get behaving filters to fully work
before the end of the year, then jan/feb/mar will be to ensure smtpd can
cope with misbheaving filters.
the areas we know need to be improved:
- all kinds of timeouts: smtp session timeout, filters timeout, ...
- all kinds of DATA issues: filter responding with end of message, while
client hasn't responded with end of message yet, ...
- all kinds of exhaustions: failure to allocate filter sessions, failure
to send data to filters because the pipe is exhausted, ...
- all kinds of filters fuckup: filters responding with bad phases or bad
sessions or bad action, etc... some are bugs, some are legit, ...
now that we know filters work, including in chains, we can focus on what
is needed to make them rock solid for April :-)
please continue raising questions because the more people play with them
the more we can spot what needs to be investigated.
--
Gilles Chehade @poolpOrg
https://www.poolp.org tip me: https://paypal.me/poolpOrg
--
You received this mail because you are subscribed to misc@opensmtpd.org
To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
