Re: [ANNOUNCE] haproxy-2.6-dev6

[William Lallemand]

On Sat, Apr 16, 2022 at 08:26:33PM +0200, Willy Tarreau wrote:
> On Sat, Apr 16, 2022 at 11:12:41PM +0500,  ??? wrote:
> > > > > William has also set up a build system that's triggered by the CI and
> > > that
> > > > > produces packages of the latest development version for various
> > > distros.
> > > > > The goal is to help users deploy development versions to participate 
> > > > > to
> > > > > the testing and benefit early from new features, as we know that till
> > > now
> > > > > it used to require particular efforts and that not everyone has enough
> > > > > time to think about rebuilding packages often. I'll let William expand
> > > on
> > > > > this point regarding what's covered and how to use this.
> > > > >
> > > >
> > > > that's interesting.
> > > > any links?
> > >
> > > As I said he will share the details :-)
> > >
> > 
> > I've something ...
> > wlallemand/haproxy-nightly-build (github.com)
> > 
> 
> Ah I forgot, he updated the links in the wiki:
> 
>  
> https://github.com/haproxy/wiki/wiki/Packages#community-maintained-repositories
> 
> But he knows the current status and the next steps if any.
> 
> Willy
> 

Indeed,

It uses the Open Build System from OpenSuse to generate packages on
debian and ubuntu. Each time we push some code on the git repository,
github trigger the build by doing an HTTP request on the service.


The status of the latest build is here: 
https://build.opensuse.org/project/show/home:wlallemand

At the moment it builds packages for Debian 10/11/Unstable and Ubuntu
20.04 with multiple architecture depending on what's available.
More distributions are available, like redhat/centos but I will need to
spend some time on this.

The packages are available here: 
https://software.opensuse.org/download/package?package=haproxy=home%3Awlallemand

You can grab them individually or install the repository, which is
really convenient if you want to update on a daily basis.


The version of the package is generated this way:

   haproxy_   2.6-dev6  .0.   git20220416  .a8b1065b6  -0+52.2_amd64.deb
 ^   ^ ^   ^  ^ 
 |   | |   |  + Some obscure OBS 
sub-version 
 |   | |   |
 |   | |   + Git Commit ID
 |   | |
 |   | + Date of the day
 |   |
 |   + Number of commits after the tag
 |
 + Latest tag

I hope this will be useful for users that want to deploy the development
version for testing purposes.

Regards,

-- 
William Lallemand

Re: [ANNOUNCE] haproxy-2.6-dev6

[Willy Tarreau]

On Sat, Apr 16, 2022 at 11:12:41PM +0500,  ??? wrote:
> > > > William has also set up a build system that's triggered by the CI and
> > that
> > > > produces packages of the latest development version for various
> > distros.
> > > > The goal is to help users deploy development versions to participate to
> > > > the testing and benefit early from new features, as we know that till
> > now
> > > > it used to require particular efforts and that not everyone has enough
> > > > time to think about rebuilding packages often. I'll let William expand
> > on
> > > > this point regarding what's covered and how to use this.
> > > >
> > >
> > > that's interesting.
> > > any links?
> >
> > As I said he will share the details :-)
> >
> 
> I've something ...
> wlallemand/haproxy-nightly-build (github.com)
> 

Ah I forgot, he updated the links in the wiki:

 
https://github.com/haproxy/wiki/wiki/Packages#community-maintained-repositories

But he knows the current status and the next steps if any.

Willy

Re: [ANNOUNCE] haproxy-2.6-dev6

[Илья Шипицин]

сб, 16 апр. 2022 г. в 22:40, Willy Tarreau :

> Hi Ilya,
>
> On Sat, Apr 16, 2022 at 10:08:58PM +0500,  ??? wrote:
> > ??, 16 ???. 2022 ?. ? 19:07, Willy Tarreau :
> >
> > > Hi,
> > >
> > > HAProxy 2.6-dev6 was released on 2022/04/16. It added 150 new commits
> > > after version 2.6-dev5.
> > >
> >
> > can we schedule those coverity findings before 2.6 ?
> >
> > src/haproxy.c: unintentional integer overflow suspected by coverity ·
> Issue
> > #1585 · haproxy/haproxy (github.com)
> > 
> > src/cfgparse.c: use after free suspected by coverity · Issue #1563 ·
> > haproxy/haproxy (github.com)
> > 
>
> Yes, they're among those linked at haproxy.org/l/code-reports and ideally
> all of them should be addressed before the release.
>
> > > William has also set up a build system that's triggered by the CI and
> that
> > > produces packages of the latest development version for various
> distros.
> > > The goal is to help users deploy development versions to participate to
> > > the testing and benefit early from new features, as we know that till
> now
> > > it used to require particular efforts and that not everyone has enough
> > > time to think about rebuilding packages often. I'll let William expand
> on
> > > this point regarding what's covered and how to use this.
> > >
> >
> > that's interesting.
> > any links?
>
> As I said he will share the details :-)
>

I've something ...
wlallemand/haproxy-nightly-build (github.com)



>
> Cheers,
> Willy
>

Re: [ANNOUNCE] haproxy-2.6-dev6

[Willy Tarreau]

Hi Ilya,

On Sat, Apr 16, 2022 at 10:08:58PM +0500,  ??? wrote:
> ??, 16 ???. 2022 ?. ? 19:07, Willy Tarreau :
> 
> > Hi,
> >
> > HAProxy 2.6-dev6 was released on 2022/04/16. It added 150 new commits
> > after version 2.6-dev5.
> >
> 
> can we schedule those coverity findings before 2.6 ?
> 
> src/haproxy.c: unintentional integer overflow suspected by coverity · Issue
> #1585 · haproxy/haproxy (github.com)
> 
> src/cfgparse.c: use after free suspected by coverity · Issue #1563 ·
> haproxy/haproxy (github.com)
> 

Yes, they're among those linked at haproxy.org/l/code-reports and ideally
all of them should be addressed before the release.

> > William has also set up a build system that's triggered by the CI and that
> > produces packages of the latest development version for various distros.
> > The goal is to help users deploy development versions to participate to
> > the testing and benefit early from new features, as we know that till now
> > it used to require particular efforts and that not everyone has enough
> > time to think about rebuilding packages often. I'll let William expand on
> > this point regarding what's covered and how to use this.
> >
> 
> that's interesting.
> any links?

As I said he will share the details :-)

Cheers,
Willy

Re: [ANNOUNCE] haproxy-2.6-dev6

[Илья Шипицин]

сб, 16 апр. 2022 г. в 19:07, Willy Tarreau :

> Hi,
>
> HAProxy 2.6-dev6 was released on 2022/04/16. It added 150 new commits
> after version 2.6-dev5.
>

can we schedule those coverity findings before 2.6 ?

src/haproxy.c: unintentional integer overflow suspected by coverity · Issue
#1585 · haproxy/haproxy (github.com)

src/cfgparse.c: use after free suspected by coverity · Issue #1563 ·
haproxy/haproxy (github.com)



>
> This release mostly focuses on integrating the second half of the merge
> of the stream interface and conn_stream that I spoke about last week, and
> it concludes this operation that was envisionned since the introduction of
> the conn_stream in 1.8. While the change is very methodic, it touches many
> places and there is a non-null risk that something was broken, hence the
> reason for exposing this rework as soon as possible. There is no expected
> change for users (aside a possible bug of course), but for developers it
> will change the way to access the lower layers from the upper ones (it will
> be simpler but for those like me who've used that since 1.4 or so, it will
> take some time to get used to it).
>
> QUIC saw a small batch of fixes and improvements (some are still pending).
> One visible part is that the SSL sample fetch functions now work on QUIC
> connections (e.g. ssl_fc or ssl_f_serial etc) and that the source address
> is now properly retrieved. The destination address is still inaccurate,
> the listener's address is retrieved (but if it's bound to an exact address
> instead of 0.0.0.0, the correct one will be reported). The reason is the
> limited API to retrieve the destination address of an incoming datagram.
> We've found a possibility to explore soon on Linux.
>
> A few TCP info sample fetch methods were enabled on MacOS.
>
> A few long-pending issues were addressed, and these fixes will be
> backported
> to affected versions, but there's nothing exceptional on this front.
>
> After some discussion with William and Emeric around the build trouble made
> by OpenSSL engines in OpenSSL 3.0 that dumps a torrent of warnings that
> hide
> important ones, and the fact that users of engines usually build some or
> all
> parts themselves, it was decided that engines are not enabled by default
> anymore, but that they may be enabled by passing "USE_ENGINE=1" to make.
> As such we now have the two following options:
>
>   - build with just USE_OPENSSL=1, engines are disabled, no warning should
> be emitted. The SSL maintainers think it should be the default since
> the future of engines in OpenSSL is uncertain due to the new
> "providers"
> API that might possibly change certain settings in the future anyway.
>
>   - build with USE_OPENSSL=1 USE_ENGINE=1 to continue to enable engines.
> In this case an extra option is passed to disable deprecation warnings
> in OpenSSL so that the build should not emit any warning either, but
> may also hide future deprecation warnings.
>
> My personal suspicion is that distros will build without engines since
> there is none that we're aware of that works out of the box without having
> at least to touch openssl a little bit, and that advanced users will
> continue to build their own optimized packages with this option enabled.
> Time will tell, as usual.
>
> Another improvement which is not related to the code, with the precious
> help of Tim and Cyril, we could finally set up an automatic generation of
> the HTML documentation. It's performed daily and published on github pages
> at http://docs.haproxy.org.
>
> William has also set up a build system that's triggered by the CI and that
> produces packages of the latest development version for various distros.
> The goal is to help users deploy development versions to participate to
> the testing and benefit early from new features, as we know that till now
> it used to require particular efforts and that not everyone has enough
> time to think about rebuilding packages often. I'll let William expand on
> this point regarding what's covered and how to use this.
>

that's interesting.
any links?


>
> Finally we've added links to remaining issues affecting the development
> versions below (verified bugs, unqualified ones and automated code
> reports).
> These are just shortcuts for filters in the issue tracker, but it's
> pleasant
> to see that there are quite few left thus we're on a good trend.
>
> LAST MINUTE:
> 
> The deployment on haproxy.org crashed during the typing of this message.
> I've pushed a fix that seems to have fixed it but I'll double-check with
> Christopher next week if I'm fixing the bug of just hiding it. Please do
> not deploy it in production before we send the signal that it's OK (we'll
> emit a new version then).
>
> Please find the usual URLs below :
>Site index   : http://www.haproxy.org/
>Documentation: 

[ANNOUNCE] haproxy-2.6-dev6

[Willy Tarreau]

Hi,

HAProxy 2.6-dev6 was released on 2022/04/16. It added 150 new commits
after version 2.6-dev5.

This release mostly focuses on integrating the second half of the merge
of the stream interface and conn_stream that I spoke about last week, and
it concludes this operation that was envisionned since the introduction of
the conn_stream in 1.8. While the change is very methodic, it touches many
places and there is a non-null risk that something was broken, hence the
reason for exposing this rework as soon as possible. There is no expected
change for users (aside a possible bug of course), but for developers it
will change the way to access the lower layers from the upper ones (it will
be simpler but for those like me who've used that since 1.4 or so, it will
take some time to get used to it).

QUIC saw a small batch of fixes and improvements (some are still pending).
One visible part is that the SSL sample fetch functions now work on QUIC
connections (e.g. ssl_fc or ssl_f_serial etc) and that the source address
is now properly retrieved. The destination address is still inaccurate,
the listener's address is retrieved (but if it's bound to an exact address
instead of 0.0.0.0, the correct one will be reported). The reason is the
limited API to retrieve the destination address of an incoming datagram.
We've found a possibility to explore soon on Linux.

A few TCP info sample fetch methods were enabled on MacOS.

A few long-pending issues were addressed, and these fixes will be backported
to affected versions, but there's nothing exceptional on this front.

After some discussion with William and Emeric around the build trouble made
by OpenSSL engines in OpenSSL 3.0 that dumps a torrent of warnings that hide
important ones, and the fact that users of engines usually build some or all
parts themselves, it was decided that engines are not enabled by default
anymore, but that they may be enabled by passing "USE_ENGINE=1" to make.
As such we now have the two following options:

  - build with just USE_OPENSSL=1, engines are disabled, no warning should
be emitted. The SSL maintainers think it should be the default since
the future of engines in OpenSSL is uncertain due to the new "providers"
API that might possibly change certain settings in the future anyway.

  - build with USE_OPENSSL=1 USE_ENGINE=1 to continue to enable engines.
In this case an extra option is passed to disable deprecation warnings
in OpenSSL so that the build should not emit any warning either, but
may also hide future deprecation warnings.

My personal suspicion is that distros will build without engines since
there is none that we're aware of that works out of the box without having
at least to touch openssl a little bit, and that advanced users will
continue to build their own optimized packages with this option enabled.
Time will tell, as usual.

Another improvement which is not related to the code, with the precious
help of Tim and Cyril, we could finally set up an automatic generation of
the HTML documentation. It's performed daily and published on github pages
at http://docs.haproxy.org.

William has also set up a build system that's triggered by the CI and that
produces packages of the latest development version for various distros.
The goal is to help users deploy development versions to participate to
the testing and benefit early from new features, as we know that till now
it used to require particular efforts and that not everyone has enough
time to think about rebuilding packages often. I'll let William expand on
this point regarding what's covered and how to use this.

Finally we've added links to remaining issues affecting the development
versions below (verified bugs, unqualified ones and automated code reports).
These are just shortcuts for filters in the issue tracker, but it's pleasant
to see that there are quite few left thus we're on a good trend.

LAST MINUTE:

The deployment on haproxy.org crashed during the typing of this message.
I've pushed a fix that seems to have fixed it but I'll double-check with
Christopher next week if I'm fixing the bug of just hiding it. Please do
not deploy it in production before we send the signal that it's OK (we'll
emit a new version then).

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Documentation: http://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.6/src/
   Git repository   : http://git.haproxy.org/git/haproxy.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy.git
   Changelog: http://www.haproxy.org/download/2.6/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: