Re: plans for 2017

2017-03-13 Thread Gilles Chehade
On Mon, Mar 13, 2017 at 08:26:14AM +0100, Jiri Navratil wrote:
> Hello Mik and Gilles,
> 
> On Sat, Mar 11, 2017 at 02:12:39PM +, Mik J wrote:
> > I've implemented a quick and dirty way to retrieve statistics with bind and 
> > spamd through snmp. I could do the same with opensmtpd but the trafic of my 
> > mail server is close from 0 at the moment.I just wanted to say that in a 
> > real production environnement monitoring is quite important for instance 
> > with snmp.
> > I don't know if other people using opensmtpd share this opinion.
> 
> I'm using opensmtpd and will welcome monitoring via snmp.
> 
> I can help with testing.
> 

If someone is willing to work on this, I'll review diffs but it's not
part of my todo list and I don't have an interest myself as it didn't
prevent me from monitoring using various existing tools. It's less of
a priority than the config change, the mta rewrite and filters as far
as I'm concerned.


> PS (next item to wish list, please): and I will also welcome fix for 
> delivering to local users, that have username with upper capitals. 
> 

Don't hold your breath, that is not going to happen anytime soon, it
isn't a bug but a design choice to fold to lowercase as it makes the
code much simpler and removes ambiguities.

OpenSMTPD is not suited for MXs running mixed-case system usernames.

-- 
Gilles Chehade

https://www.poolp.org  @poolpOrg

-- 
You received this mail because you are subscribed to misc@opensmtpd.org
To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org



Re: plans for 2017

2017-03-13 Thread Jiri Navratil
Hello Mik and Gilles,

On Sat, Mar 11, 2017 at 02:12:39PM +, Mik J wrote:
> I've implemented a quick and dirty way to retrieve statistics with bind and 
> spamd through snmp. I could do the same with opensmtpd but the trafic of my 
> mail server is close from 0 at the moment.I just wanted to say that in a real 
> production environnement monitoring is quite important for instance with snmp.
> I don't know if other people using opensmtpd share this opinion.

I'm using opensmtpd and will welcome monitoring via snmp.

I can help with testing.

PS (next item to wish list, please): and I will also welcome fix for delivering 
to local users, that have username with upper capitals. 

Best regards,
Jiří

> 
> 
> Le Jeudi 9 février 2017 11h26, Gilles Chehade <gil...@poolp.org> a écrit :
>  
> 
>  On Thu, Feb 09, 2017 at 10:48:14AM +0100, Mischa wrote:
> > Hi Gilles,
> > 
> > Thank you for expressing your plans. Looking forward to the changes.
> > Keep it coming, you are doing great things!
> > 
> 
> Thanks
> 
> Also, when we've made a bit of progress, we're going to explain a bit
> more where we're going with the filters, the goal is not to keep it a
> secret until last day but to allow us to move forward without all the
> noise that would happen from the "i'd do it differently" people ;-)
> 
> Regarding the MTA changes we now exactly what we want to do but there
> is a bit of a chicken & the egg issue with the last changes that were
> mentionned. The idea is that we can achieve an MTA layer implem which
> is isofunctionnal to the current one with most of the complexity that
> is currently taking charge of optimizing routing, reusing connections
> and managing limits entirely gone. This will not only improve quality
> but also allow for new features which are painful to implement today,
> as they require touching a very tricky brick of code.
> 
> Regarding the later changes all I can say for now is that it is going
> to imply a configuration file format change, we'll probably find ways
> to retain some syntaxic sugar but we're essentially going to have the
> envelope template (the accept part) decorrelated from the action (the
> deliver to / relay part) which seems like an innocent change but will
> have (GOOD) implications on pretty much *every* layer of the daemon.
> 
> Now i'm done with the explaining, still swamped for a few days and I
> will dive back into the code.
> 
> Gilles
> 
> 
> > > On 9 Feb 2017, at 10:44, Gilles Chehade <gil...@poolp.org> wrote:
> > > 
> > > Hello misc@,
> > > 
> > > It's been calm for a while due to "real-life (tm)" events that had
> > > to be handled in priority as far as I'm concerned, I don't know of
> > > the reasons why the others are slacking though :-)
> > > 
> > > I've been willing to send this mail for a while to outline some of
> > > the big plans for 2017 regarding OpenSMTPD and some of the changes
> > > that are planned in different parts of the daemon.
> > > 
> > > 
> > > 
> > > First of all, regarding filters, since that's the question that is
> > > coming the more often:
> > > 
> > > Filters are neither dead or alive.
> > > We have implemented an API and the mechanics to make that API work
> > > and this is what people started using while we warned them not to.
> > > 
> > > Turns out that while implementing a specific filter I hit an issue
> > > which made it clear that there was a fundamental design issue with
> > > the mechanics below the API that couldn't be worked around without
> > > requiring a non-trivial refactor.
> > > 
> > > We had a long chat with eric@ about this design issue and how this
> > > could be redesigned in a way that all the work we've done is still
> > > usable and we figured a way which will reuse a big part of what we
> > > already did, which guarantees that we will not find a design error
> > > later down the chain and which as a bonus simplifies the daemon.
> > > 
> > > We're going to be working towards this way but now that we have an
> > > experience in how providing the code early turned into a nightmare
> > > for me, we'll work in a private branch then show the diff when the
> > > code is working enough that it can be part of snapshots :-)
> > > 
> > > 
> > > 
> > > Then, regarding the MTA we're going to do a pass of simplification
> > > because the code has evolved into something quite complex and from
> > > experience gathered in the mail industry th

Re: plans for 2017

2017-03-11 Thread Mik J
Hello Gilles,
Thanks too for sharing.
I've implemented a quick and dirty way to retrieve statistics with bind and 
spamd through snmp. I could do the same with opensmtpd but the trafic of my 
mail server is close from 0 at the moment.I just wanted to say that in a real 
production environnement monitoring is quite important for instance with snmp.
I don't know if other people using opensmtpd share this opinion.
However, thank you all for the good work


Le Jeudi 9 février 2017 11h26, Gilles Chehade <gil...@poolp.org> a écrit :
 

 On Thu, Feb 09, 2017 at 10:48:14AM +0100, Mischa wrote:
> Hi Gilles,
> 
> Thank you for expressing your plans. Looking forward to the changes.
> Keep it coming, you are doing great things!
> 

Thanks

Also, when we've made a bit of progress, we're going to explain a bit
more where we're going with the filters, the goal is not to keep it a
secret until last day but to allow us to move forward without all the
noise that would happen from the "i'd do it differently" people ;-)

Regarding the MTA changes we now exactly what we want to do but there
is a bit of a chicken & the egg issue with the last changes that were
mentionned. The idea is that we can achieve an MTA layer implem which
is isofunctionnal to the current one with most of the complexity that
is currently taking charge of optimizing routing, reusing connections
and managing limits entirely gone. This will not only improve quality
but also allow for new features which are painful to implement today,
as they require touching a very tricky brick of code.

Regarding the later changes all I can say for now is that it is going
to imply a configuration file format change, we'll probably find ways
to retain some syntaxic sugar but we're essentially going to have the
envelope template (the accept part) decorrelated from the action (the
deliver to / relay part) which seems like an innocent change but will
have (GOOD) implications on pretty much *every* layer of the daemon.

Now i'm done with the explaining, still swamped for a few days and I
will dive back into the code.

Gilles


> > On 9 Feb 2017, at 10:44, Gilles Chehade <gil...@poolp.org> wrote:
> > 
> > Hello misc@,
> > 
> > It's been calm for a while due to "real-life (tm)" events that had
> > to be handled in priority as far as I'm concerned, I don't know of
> > the reasons why the others are slacking though :-)
> > 
> > I've been willing to send this mail for a while to outline some of
> > the big plans for 2017 regarding OpenSMTPD and some of the changes
> > that are planned in different parts of the daemon.
> > 
> > 
> > 
> > First of all, regarding filters, since that's the question that is
> > coming the more often:
> > 
> > Filters are neither dead or alive.
> > We have implemented an API and the mechanics to make that API work
> > and this is what people started using while we warned them not to.
> > 
> > Turns out that while implementing a specific filter I hit an issue
> > which made it clear that there was a fundamental design issue with
> > the mechanics below the API that couldn't be worked around without
> > requiring a non-trivial refactor.
> > 
> > We had a long chat with eric@ about this design issue and how this
> > could be redesigned in a way that all the work we've done is still
> > usable and we figured a way which will reuse a big part of what we
> > already did, which guarantees that we will not find a design error
> > later down the chain and which as a bonus simplifies the daemon.
> > 
> > We're going to be working towards this way but now that we have an
> > experience in how providing the code early turned into a nightmare
> > for me, we'll work in a private branch then show the diff when the
> > code is working enough that it can be part of snapshots :-)
> > 
> > 
> > 
> > Then, regarding the MTA we're going to do a pass of simplification
> > because the code has evolved into something quite complex and from
> > experience gathered in the mail industry these last few years, the
> > code can be made much more efficient while MUCH simpler.
> > 
> > 
> > 
> > Finally, there is ongoing work that's going to span over months to
> > improve some configuration structures which is going to have a lot
> > of interesting side-effects which I'm going to keep as a surprise,
> > but that are going to be impressive. I personnally look forward to
> > this more than filters given the amounts of improvements this will
> > unlock in many areas ranging from configuration, to reload, to MTA
> > and MDA.
> > 
> > 
> > Stay tuned !
> > 
> > 
> > -- 
> &

Re: plans for 2017

2017-02-09 Thread Gilles Chehade
On Thu, Feb 09, 2017 at 10:48:14AM +0100, Mischa wrote:
> Hi Gilles,
> 
> Thank you for expressing your plans. Looking forward to the changes.
> Keep it coming, you are doing great things!
> 

Thanks

Also, when we've made a bit of progress, we're going to explain a bit
more where we're going with the filters, the goal is not to keep it a
secret until last day but to allow us to move forward without all the
noise that would happen from the "i'd do it differently" people ;-)

Regarding the MTA changes we now exactly what we want to do but there
is a bit of a chicken & the egg issue with the last changes that were
mentionned. The idea is that we can achieve an MTA layer implem which
is isofunctionnal to the current one with most of the complexity that
is currently taking charge of optimizing routing, reusing connections
and managing limits entirely gone. This will not only improve quality
but also allow for new features which are painful to implement today,
as they require touching a very tricky brick of code.

Regarding the later changes all I can say for now is that it is going
to imply a configuration file format change, we'll probably find ways
to retain some syntaxic sugar but we're essentially going to have the
envelope template (the accept part) decorrelated from the action (the
deliver to / relay part) which seems like an innocent change but will
have (GOOD) implications on pretty much *every* layer of the daemon.

Now i'm done with the explaining, still swamped for a few days and I
will dive back into the code.

Gilles


> > On 9 Feb 2017, at 10:44, Gilles Chehade <gil...@poolp.org> wrote:
> > 
> > Hello misc@,
> > 
> > It's been calm for a while due to "real-life (tm)" events that had
> > to be handled in priority as far as I'm concerned, I don't know of
> > the reasons why the others are slacking though :-)
> > 
> > I've been willing to send this mail for a while to outline some of
> > the big plans for 2017 regarding OpenSMTPD and some of the changes
> > that are planned in different parts of the daemon.
> > 
> > 
> > 
> > First of all, regarding filters, since that's the question that is
> > coming the more often:
> > 
> > Filters are neither dead or alive.
> > We have implemented an API and the mechanics to make that API work
> > and this is what people started using while we warned them not to.
> > 
> > Turns out that while implementing a specific filter I hit an issue
> > which made it clear that there was a fundamental design issue with
> > the mechanics below the API that couldn't be worked around without
> > requiring a non-trivial refactor.
> > 
> > We had a long chat with eric@ about this design issue and how this
> > could be redesigned in a way that all the work we've done is still
> > usable and we figured a way which will reuse a big part of what we
> > already did, which guarantees that we will not find a design error
> > later down the chain and which as a bonus simplifies the daemon.
> > 
> > We're going to be working towards this way but now that we have an
> > experience in how providing the code early turned into a nightmare
> > for me, we'll work in a private branch then show the diff when the
> > code is working enough that it can be part of snapshots :-)
> > 
> > 
> > 
> > Then, regarding the MTA we're going to do a pass of simplification
> > because the code has evolved into something quite complex and from
> > experience gathered in the mail industry these last few years, the
> > code can be made much more efficient while MUCH simpler.
> > 
> > 
> > 
> > Finally, there is ongoing work that's going to span over months to
> > improve some configuration structures which is going to have a lot
> > of interesting side-effects which I'm going to keep as a surprise,
> > but that are going to be impressive. I personnally look forward to
> > this more than filters given the amounts of improvements this will
> > unlock in many areas ranging from configuration, to reload, to MTA
> > and MDA.
> > 
> > 
> > Stay tuned !
> > 
> > 
> > -- 
> > Gilles Chehade
> > 
> > https://www.poolp.org  @poolpOrg
> > 
> > -- 
> > You received this mail because you are subscribed to misc@opensmtpd.org
> > To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
> > 
> 
> 
> -- 
> You received this mail because you are subscribed to misc@opensmtpd.org
> To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
> 

-- 
Gilles Chehade

https://www.poolp.org  @poolpOrg

-- 
You received this mail because you are subscribed to misc@opensmtpd.org
To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org



Re: plans for 2017

2017-02-09 Thread Mischa
Hi Gilles,

Thank you for expressing your plans. Looking forward to the changes.
Keep it coming, you are doing great things!

Mischa

> On 9 Feb 2017, at 10:44, Gilles Chehade <gil...@poolp.org> wrote:
> 
> Hello misc@,
> 
> It's been calm for a while due to "real-life (tm)" events that had
> to be handled in priority as far as I'm concerned, I don't know of
> the reasons why the others are slacking though :-)
> 
> I've been willing to send this mail for a while to outline some of
> the big plans for 2017 regarding OpenSMTPD and some of the changes
> that are planned in different parts of the daemon.
> 
> 
> 
> First of all, regarding filters, since that's the question that is
> coming the more often:
> 
> Filters are neither dead or alive.
> We have implemented an API and the mechanics to make that API work
> and this is what people started using while we warned them not to.
> 
> Turns out that while implementing a specific filter I hit an issue
> which made it clear that there was a fundamental design issue with
> the mechanics below the API that couldn't be worked around without
> requiring a non-trivial refactor.
> 
> We had a long chat with eric@ about this design issue and how this
> could be redesigned in a way that all the work we've done is still
> usable and we figured a way which will reuse a big part of what we
> already did, which guarantees that we will not find a design error
> later down the chain and which as a bonus simplifies the daemon.
> 
> We're going to be working towards this way but now that we have an
> experience in how providing the code early turned into a nightmare
> for me, we'll work in a private branch then show the diff when the
> code is working enough that it can be part of snapshots :-)
> 
> 
> 
> Then, regarding the MTA we're going to do a pass of simplification
> because the code has evolved into something quite complex and from
> experience gathered in the mail industry these last few years, the
> code can be made much more efficient while MUCH simpler.
> 
> 
> 
> Finally, there is ongoing work that's going to span over months to
> improve some configuration structures which is going to have a lot
> of interesting side-effects which I'm going to keep as a surprise,
> but that are going to be impressive. I personnally look forward to
> this more than filters given the amounts of improvements this will
> unlock in many areas ranging from configuration, to reload, to MTA
> and MDA.
> 
> 
> Stay tuned !
> 
> 
> -- 
> Gilles Chehade
> 
> https://www.poolp.org  @poolpOrg
> 
> -- 
> You received this mail because you are subscribed to misc@opensmtpd.org
> To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org
> 


-- 
You received this mail because you are subscribed to misc@opensmtpd.org
To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org



plans for 2017

2017-02-09 Thread Gilles Chehade
Hello misc@,

It's been calm for a while due to "real-life (tm)" events that had
to be handled in priority as far as I'm concerned, I don't know of
the reasons why the others are slacking though :-)

I've been willing to send this mail for a while to outline some of
the big plans for 2017 regarding OpenSMTPD and some of the changes
that are planned in different parts of the daemon.



First of all, regarding filters, since that's the question that is
coming the more often:

Filters are neither dead or alive.
We have implemented an API and the mechanics to make that API work
and this is what people started using while we warned them not to.

Turns out that while implementing a specific filter I hit an issue
which made it clear that there was a fundamental design issue with
the mechanics below the API that couldn't be worked around without
requiring a non-trivial refactor.

We had a long chat with eric@ about this design issue and how this
could be redesigned in a way that all the work we've done is still
usable and we figured a way which will reuse a big part of what we
already did, which guarantees that we will not find a design error
later down the chain and which as a bonus simplifies the daemon.

We're going to be working towards this way but now that we have an
experience in how providing the code early turned into a nightmare
for me, we'll work in a private branch then show the diff when the
code is working enough that it can be part of snapshots :-)



Then, regarding the MTA we're going to do a pass of simplification
because the code has evolved into something quite complex and from
experience gathered in the mail industry these last few years, the
code can be made much more efficient while MUCH simpler.



Finally, there is ongoing work that's going to span over months to
improve some configuration structures which is going to have a lot
of interesting side-effects which I'm going to keep as a surprise,
but that are going to be impressive. I personnally look forward to
this more than filters given the amounts of improvements this will
unlock in many areas ranging from configuration, to reload, to MTA
and MDA.


Stay tuned !


-- 
Gilles Chehade

https://www.poolp.org  @poolpOrg

-- 
You received this mail because you are subscribed to misc@opensmtpd.org
To unsubscribe, send a mail to: misc+unsubscr...@opensmtpd.org