Hi Willy.
Finally I found the time to write a answer.
On 25/05/2018 18:10, Willy Tarreau wrote:
Hi all,
I've long wanted to send a status update on where haproxy 1.9 is
going, and seeing some recent threads speculating about what will be
available reminded me that it's really time to send this update. Be
careful, this e-mail is long.
8-O yes but very informative. Thanks for writing.
[snipp]
- make a "resolvers" section able to use /etc/resolv.conf [Ben]. Baptiste
is going to take a look at this very soon, I think we should be good
soon on this one.
Are there any plans for DoH (= https://github.com/curl/curl/wiki/DNS-over-HTTPS)
[snipp]
- native HTTP representation allowing end-to-end HTTP/2
[Christopher] : this will allow HTTP/2 to be used end-to-end. For
now Chris is working on the "easy" part which is the HTTP/1
parsing/conversion. I'm putting quotes around "easy" because while
HTTP/1 appears easy to deal with, that's where the highest number
of obstacles are present due to the fact that HTTP/1 is currently
used as the native internal representation (H2 is currently
translated to H1). This will heavily rely on the mux changes. We
know that some parts ("tcp-request content" L7-based rules, option
http-send-name-header, and filters/compression) will cause some
head scratching. The expected gains from this part are
tremendous. But while I thought that H2 and threads used to be the
most complex changes over the last decade, I think this one's
complexity has already surpassed them both. I'd say the chances of
success for 1.9 are around 70%, which is not much but which still
gives us the motivation for pursuing the efforts. Once completed,
Christopher will deserve a barrel of beer to forget about this
painful task ;-)
Full ack.
That's one part for the gRPC, as Daniel mentioned in a previous mail ;-)
Will this be a 'plugin'?
The idea behind this question is to add another protocol plugin like MQTT
or other protocols
[snipp]
- SSL layer splitting between FD level and buffer level [Olivier] :
this will be required for QUIC so that we can feed buffers
containing SSL traffic to OpenSSL. For now the SSL layer relies on
OpenSSL's native read()/write()
calls (yes, the ones that force you to disable SIGPIPE in gdb).
This is one question I wanted to ask 'what's the plan for QIUC is.'
Sounds like it's on the roadmap.
Some ideas to study later, nothing really assigned :
- certificates : study if it makes sense to parse them on the fly on
first use to also save on startup time.
- logs: some improvements to the logging system have been discussed
about a few times, which can be summarized like this :
- some people would like a set of log "profiles" that can be
reused everywhere instead of copy-pasting the same log-format
lines.
- some people really need a per-server log-format because they
send, say, native logs locally and JSON logs to another
server.
- some people want to load-balance between log servers
- other people want to sample logs without having to trick the
config
=> all of this has to be addressed together.
* How about to let rsyslog handle this. There are queues and pools and
ratelimits also are there some recipes how to create json logs.
or
* How about to build a dedicated worker for this like SPOE which handles
the specific logging requirement?
Jm2C
- the "return" directive, regularly talked about, may possible be done
for 1.9.
8-O What's this?
[snipp]
Last point, as some of you might have noticed, we finally recovered
the haproxy GitHub account (thanks to Jeff for this as well as Dan,
Lukas and Joe for helping coordinate the operations). A new haproxy
repository was placed there only with mainline for now. By the way if
you're having a fork from more than one month ago, chances are that
you accidently forked the wrong one ;-)
We do have a few plans with this account now, some of which seem
appealing, and others who don't sound really compatible with some
annoying GitHub limitations, making me think that maybe in the end we
should switch to Gitlab (William already created haproxy.org there) :
OH I like gitlab much more then github ;-)
- re-enable issues : we still don't have a public bug tracker/todo
list and it's a pain for everyone. It would be nice to show what's
pending or being worked on, and to sometimes add extra info
regarding a given task. The problem we've faced in the past with
GitHub's issue tracker is that it's impossible to restrict the
creation of new issues to participants. Given that haproxy is a
complex component, the boundary between human error,
misunderstanding and bug is extremely thin. It resulted in the
issue tracker being filled with wrong bugs, 100% of which were in
fact requests for help. It makes the utility totally useless for
development and bug fixing as it requires more time to maintain in
a clean state than it takes to put issues in a mailbox. Some
people suggested using issue templates, but I hardly see how this
will help, at best it will annoy regular participants, at worst it
will not stop polluters from filling crap there. What I'd ideally
like would be that about any participant here on the list
(i.e. those who help others) can create an issue, and that anyone
could see open issues and add info to existing issues.
The closest we've found to this is Gitlab, but when issue creation
is limited to contributors, only these ones can watch them. I tend
to consider that once many participants here are contributors, the
tracker becomes almost public thus this limitation is not a big
deal, but it's still annoying.
A limitation that isn't addressed by any of them is that an issue
has a single status and not one per maintenance branch. Some will
say that labels replace this but I'd say that this isn't true, it
just vaguely emulates this. Anyway if we don't have better we can
go with this. I often dream of the day someone pissed of by using
prehistoric issue trackers writes a modern one, just like when
Linus created Git...
In addition to Tims statement I suggest that we can use some labels to
separate the issue type like, help, lua, core, and so on. I think we
can use similar labels as we already use in the patches/changelog
[DOC/LUA/SSL...].
With labels can everyone just limit the view which the person want.
- wiki : we all know that the architecture guide is obsolete,
everyone wants to refresh it and nobody can because it's a tedious
task that no single person can address, and nobody anymore knows
all haproxy pieces. In addition a lot of participants have useful
tips to share and would be much more at ease with putting this
into a wiki than by editing architecture.txt. Thus I'd like to
reuse either Github or Gitlab's wiki to place real contents edited
and maintained by the community, and to kill this obsolete
file. This way the obsolete stuff on the main haproxy.org page
could go as well.
I agree here with Tim that the *pages solution looks much more usable
then the wiki way.
As I build my homepage www.me2digital.com with gohugo it would be nice
to have such a solution here also in place.
https://gohugo.io/hosting-and-deployment/hosting-on-github/
I can lend you a hand on this, at least for the beginning.
- automated builds and CI : gitlab and github have these
possibilities (either natively or relying on third party
products). It would help us detect the occasional stupid build
breakage (like building with/ without threads, with/without SSL).
Oh yes this would be nice but it's a beast on there own.
- we could imagine reusing self-hosted pages to put Cyril's
generated docs, or to link there if it's easier. Similar for
Thierry's Lua docs, it will first depend on what these ones prefer
(I don't want to steal your work guys, don't worry, just
suggesting that it could be done if you find it desirable).
As Cyril's docs are already on "github pages" should the migration be a
easy task, IMHO.
- maybe placing the releases there will help (I don't know the
upload process for any of these, will have to check closer). It
may even be used for Aleks' docker images if that makes sense at
all (I don't know).
I don't think that the docker images are good releases on github, But
I'm open for discussion if it makes sense.
The doc suggest that the releases can be "easily build" but I haven't
done this before so maybe some one on the list with more experience could
share the experience with us.
https://help.github.com/articles/about-releases/
Feel free to discuss these points here on the list as usual. Please don't
PM me, I don't have time to rehash the same points with multiple persons.
If I missed anything important that you care about, say it now.
Thank you for writing this mail too keep us up2date.
thanks,
Willy
Very best regars
Aleks
