Hi all,
I would have liked to send this ealier but 1.8 kept us quite busy for a
while.
Last year's development cycle went impressively well in my opinion, with
multiple teams being able to work in parallel without stepping too much
onto each others' toes, and bug fixes being batched thanks to the awesome
help from some members here and on discourse. So I'd like to stick to
that model for this development cycle, still with the same flexibility
we had during the previous cycle.
Last year I said that the dev cycle would be cut in 3 phases with the
first one ending at a strict date. In fact we never respect dates, but
they are still useful to fix a direction and let people know whether
they're about to be late or they are really too late.
Thus I'd like that all those willing to work on something announce it
before end of March so that we know who does what. After this, no more
unplanned important merges, but we can possibly decide to open a -next
branch if needed. We should start to emit -rc1 around end of September,
marking the freeze of any development, and plan for a release between
mid-October and end of November. In an ideal world we would stop any
dev before the summer holiday, but we tried this many times in the
past and it never worked, it always shifts.
For now we have two big changes scheduled in parallel, that are absolutely
required to pursue the work done on H2 so that we can do end-to-end H2 :
- internal HTTP representation : instead of translating H2 to H1 at
the edge, we'll translate H2 and H1 to "HTTP". This will allow us
not to lose any semantics due to the representation (eg: the "never
indexed" header fields), and to clean up a lot of the internal API.
It should also improve our ability to have a properly working
"http-send-name-header", and to simplify everything related to HTTP
header manipulation (no more memmove, possibly a better interfacing
with Lua, etc). Christopher has already started taking a look at this
and I'm withing him good luck ;-)
- rearchitecture of the connection layers. The current architecture we
have dates from 2012 now and was made to support SSL. But with QUIC
at the corner, a file decriptor oriented I/O scheduler is no more
future-proof and we've already met some difficulties in H2 that
justify breaking this into pieces that stack much better, and hopefully
more independantly of the threads. This also directly involves some
scheduler updates. Olivier has started working on this, with already
a first success in removing the locks from the FD cache, bringing a
60% performance increase at 12 threads. This and the rework above will
cause a lot of pain at merge time...
In addition to this, William has identified a number of improvements to
make to the cache, that he's willing to work on, as well as to improve
the way the SSL certificates are managed (ie: split crt vs key, and merge
same ones together in memory to save RAM for those dealing with millions
of certs in memory). Fred plans to bring SSL support to the peers among
other things, and is working on a regression testing suite (yeah!).
Among the cool stuff to do that is not assigned yet, we know that the
health checks should be improved (eg: ability to add a header after a
POST request). Probably that other things need to be discussed regarding
health checks. We just need to be careful that such changes don't collide
too much with the work above.
It would also be cool to exploit the peers a bit more. In the past we
imagined being able to list them with their state on the CLI, use
their state to divide a server's "maxconn" setting by #peers+1 when a
"shared-with <peers>" directive is present so that each maxconn
automatically adapts depending on the number of the visible peers.
We thought about a "debug" converter which would dump the sample into
an internal ring buffer that could be consulted from the CLI using
"show debug-ring" for example. That could also help collect information
on line number of matched ACLs.
A number of people asked for a per-server log-format, possibly coupled
with the ability to define log-format profiles in their own sections,
that could be reused at various places. I'm not aware of anyone working
on this, feel free to volunteer.
I have identified a number of non-user-visible stuff to rework internally
such as unifying chunks and buffers, but that's part of the permanent
maintenance, cleanup and update of the internal infrastructure. I'd also
like to support an option on the CLI to mask some fields so that bug
reporters could more easily anonymize their output when we ask them to
do so. I started to look into the code for this and it would benefit a
lot from the aforementionned change (merging chunks and buffers).
Some may have other ideas. Please keep in mind that it's nice to put a
wish list, but it's much better to be able to do certain things by
oneself. It may seem difficult first but once you start you'll figure
it is not, and that people here on the list are helpful. If you think
your C is a bit rusted, you probably are a good candidate to try to
develop something useful to everyone, because you'll read more than
you'll write and you'll end up doing a good job!
Oh, before I forget, since nobody asked for 1.4 to continue to be
maintained, I've just marked it "unmaintained", and 1.5 now entered
the "critical fixes only" status. 1.4 will have lived almost 8 years
(1.4.0 was released on 2010-02-26). Given that it doesn't support
SSL, it's unlikely to be found exposed to HTTP traffic in sensitive
places anymore. If you still use it, there's nothing wrong for now,
as it's been one of the most stable versions of all times. But please
at least regularly watch the activity on the newer ones and consider
upgrading it once you see that some issues might affect it. For those
who can really not risk to face a bug, 1.6 is a very good candidate
now and is still well supported 2 years after its birth.
Cheers,
Willy
