Re: disallow HTTP 0.9 by default?

[Luca Toscano]

On Wed, Jul 21, 2021 at 10:04 PM Eric Covener  wrote:
>
> I was chasing an unrelated thread about close_notify alerts and
> reminded me -- is it time to change the default for
> HttpProtocolOptions from Allow0.9 to Require1.0?
>
> As the manual says, the requirement was dropped in RFC 7230. It seems
> like the kind of potential gadget in future desynch/smuggling kind of
> attacks that shouldn't be on by default today.
>
> Any opinions?

+1

Re: Contribution to Apache HTTPD

[Luca Toscano]

Hi Nikita,

On Thu, May 27, 2021 at 2:59 PM Nikita Popov  wrote:
>
> Hello. I'm a senior C developer working for CloudLinux. I would like
> to participate in the HTTPD project. Are there any unassigned tasks
> which I can take? And how to proceed with it? Thanks in advance.

Welcome! As Daniel suggested there are some resources to read in the
httpd website, and then you can probably start from [1] and pick up
any bug/report that you find interesting. People will be happy to
review patches in the bugzilla tasks and provide feedback!

Luca

[1]: 
https://bz.apache.org/bugzilla/buglist.cgi?chfieldto=Now=Apache%20httpd-2

Re: [VOTE] Release httpd-2.4.46

[Luca Toscano]

On Sat, Aug 1, 2020 at 4:14 PM Daniel Ruggeri  wrote:
>
> Hi, all;
>Third time is a charm! Please find below the proposed release tarball
> and signatures:
> https://dist.apache.org/repos/dist/dev/httpd/
>
> I would like to call a VOTE over the next few days to release this
> candidate tarball as 2.4.46:
> [x] +1: It's not just good, it's good enough!

Tested on Debian 10 "Buster", apr 1.6 + apr-util 1.6, openssl 1.1, php-fpm 7.3.
Verified all signatures and digests (nit: do we still need to publish
the .md5 ones?)

Thanks Daniel!

Luca

Re: svn commit: r1879641 - in /httpd/httpd/branches/2.4.x: ./ CHANGES STATUS server/util_script.c

[Luca Toscano]

Hi Ruediger,

On Wed, Jul 15, 2020 at 10:04 AM Ruediger Pluem  wrote:
>
>
> On 7/15/20 9:06 AM, Luca Toscano wrote:
> > Hi everybody,
> >
> > getting back on this. I checked mergeinfo and r1879253 r1879348 are
> > not listed, what is the best path forward to fix this? (Asking because
> > I have never merged from trunk, and I am not sure what is the best
> > path forward).
>
> Try
>
> svn merge --record-only -c 1879253,1879348 
> https://svn.apache.org/repos/asf/httpd/httpd/trunk .
>
> in a freshly svn up'ed working copy of 2.4.x.

Thanks for the answer. IIUC the above command takes care of updating
mergeinfo with the missing revs, but the problem is (I think) that
they didn't get merged in the first place along with the others (so
they are missing from 2.4.x). Since I never merged commits into 2.4.x,
I was wondering what to do to avoid messing up the state :)

Luca

Re: svn commit: r1879641 - in /httpd/httpd/branches/2.4.x: ./ CHANGES STATUS server/util_script.c

[Luca Toscano]

Hi everybody,

getting back on this. I checked mergeinfo and r1879253 r1879348 are
not listed, what is the best path forward to fix this? (Asking because
I have never merged from trunk, and I am not sure what is the best
path forward).

Thanks in advance,

Luca

On Wed, Jul 8, 2020 at 11:20 PM Luca Toscano  wrote:
>
> Hi Christophe,
>
> Thanks a lot for spotting this, good catch. I haven't proposed a
> backport in months and the first one ends up in a little mess, really
> sorry :(
>
> So judging from the diff, I think that it is missing the last two commits:
>
> http://svn.apache.org/viewvc?view=revision=1879253
> http://svn.apache.org/viewvc?view=revision=1879348
>
> The svn merge command contains a typo (didn't see it when committing),
> namely r1879348 at the end (extra r), maybe this was the issue?
>
> svn merge -c 
> 1748379,1750747,1750749,1750953,1751138,1751139,1751139,1757818,1879253,r1879348
> ^/httpd/httpd/trunk .
>
> Luca
>
> On Wed, Jul 8, 2020 at 10:11 PM Christophe JAILLET
>  wrote:
> >
> > Le 08/07/2020 à 13:39, minf...@apache.org a écrit :
> > > Author: minfrin
> > > Date: Wed Jul  8 11:39:12 2020
> > > New Revision: 1879641
> > >
> > > URL: http://svn.apache.org/viewvc?rev=1879641=rev
> > > Log:
> > >*) core: Drop an invalid Last-Modified header value coming
> > >   from a (F)CGI script instead of replacing it with Unix epoch.
> > >   Warn the users about Last-Modified header value replacements
> > >   and violations of the RFC.
> > >   trunk patch: http://svn.apache.org/r1748379
> > >http://svn.apache.org/r1750747
> > >http://svn.apache.org/r1750749
> > >http://svn.apache.org/r1750953
> > >http://svn.apache.org/r1751138
> > >http://svn.apache.org/r1751139
> > >http://svn.apache.org/r1751147
> > >http://svn.apache.org/r1757818
> > >http://svn.apache.org/r1879253
> > >http://svn.apache.org/r1879348
> > >   2.4.x: trunk patches work, final view:
> > >  
> > > http://home.apache.org/~elukey/httpd-2.4.x-core-last_modified_tz_logging.patch
> > >  svn merge -c 
> > > 1748379,1750747,1750749,1750953,1751138,1751139,1751139,1757818,1879253,r1879348
> > >  ^/httpd/httpd/trunk .
> > >   The code has been tested with a simple PHP script returning 
> > > different Last-Modified
> > >   headers (GMT now, GMT now Europe/Paris, GMT tomorrow, GMT 
> > > yesterday, PST now).
> > >   +1: elukey, jorton, jim
> > >   jorton: +1 though I'd say log at WARN or INFO for the APR_BAD_DATE 
> > > case
> > >   rather than "silently" (at normal log-level) dropping the 
> > > parsed header?
> > >   [also nit: wrapping a lone ap_log_rerror(,APLOG_X) call in
> > >   if (APLOGrX(..) is unnecessary/redundant]
> > >
> > > Modified:
> > >  httpd/httpd/branches/2.4.x/   (props changed)
> > >  httpd/httpd/branches/2.4.x/CHANGES
> > >  httpd/httpd/branches/2.4.x/STATUS
> > >  httpd/httpd/branches/2.4.x/server/util_script.c
> > >
> > > Propchange: httpd/httpd/branches/2.4.x/
> > > --
> > >Merged /httpd/httpd/trunk:r1748379
> > >
> > > Modified: httpd/httpd/branches/2.4.x/CHANGES
> > > URL: 
> > > http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/CHANGES?rev=1879641=1879640=1879641=diff
> > > ==
> > > --- httpd/httpd/branches/2.4.x/CHANGES [utf-8] (original)
> > > +++ httpd/httpd/branches/2.4.x/CHANGES [utf-8] Wed Jul  8 11:39:12 2020
> > > @@ -1,6 +1,10 @@
> > >-*- coding: 
> > > utf-8 -*-
> > >   Changes with Apache 2.4.44
> > >
> > > +  *) core: Drop an invalid Last-Modified header value coming
> > > + from a FCGI/CGI script instead of replacing it with Unix epoch.
> > > + [Luca Toscano]
> > > +
> > > *) Add support for strict content-length parsing through addition of
> > >ap_parse_strict_length() [Yann Ylavic]
> > >
> > >
> > > Modified: httpd/httpd/branches/2.4.x/STATUS
> > > URL: 
> > >

Re: svn commit: r1879641 - in /httpd/httpd/branches/2.4.x: ./ CHANGES STATUS server/util_script.c

[Luca Toscano]

Hi Christophe,

Thanks a lot for spotting this, good catch. I haven't proposed a
backport in months and the first one ends up in a little mess, really
sorry :(

So judging from the diff, I think that it is missing the last two commits:

http://svn.apache.org/viewvc?view=revision=1879253
http://svn.apache.org/viewvc?view=revision=1879348

The svn merge command contains a typo (didn't see it when committing),
namely r1879348 at the end (extra r), maybe this was the issue?

svn merge -c 
1748379,1750747,1750749,1750953,1751138,1751139,1751139,1757818,1879253,r1879348
^/httpd/httpd/trunk .

Luca

On Wed, Jul 8, 2020 at 10:11 PM Christophe JAILLET
 wrote:
>
> Le 08/07/2020 à 13:39, minf...@apache.org a écrit :
> > Author: minfrin
> > Date: Wed Jul  8 11:39:12 2020
> > New Revision: 1879641
> >
> > URL: http://svn.apache.org/viewvc?rev=1879641=rev
> > Log:
> >*) core: Drop an invalid Last-Modified header value coming
> >   from a (F)CGI script instead of replacing it with Unix epoch.
> >   Warn the users about Last-Modified header value replacements
> >   and violations of the RFC.
> >   trunk patch: http://svn.apache.org/r1748379
> >http://svn.apache.org/r1750747
> >http://svn.apache.org/r1750749
> >http://svn.apache.org/r1750953
> >http://svn.apache.org/r1751138
> >http://svn.apache.org/r1751139
> >http://svn.apache.org/r1751147
> >http://svn.apache.org/r1757818
> >http://svn.apache.org/r1879253
> >http://svn.apache.org/r1879348
> >   2.4.x: trunk patches work, final view:
> >  
> > http://home.apache.org/~elukey/httpd-2.4.x-core-last_modified_tz_logging.patch
> >  svn merge -c 
> > 1748379,1750747,1750749,1750953,1751138,1751139,1751139,1757818,1879253,r1879348
> >  ^/httpd/httpd/trunk .
> >   The code has been tested with a simple PHP script returning different 
> > Last-Modified
> >   headers (GMT now, GMT now Europe/Paris, GMT tomorrow, GMT yesterday, 
> > PST now).
> >   +1: elukey, jorton, jim
> >   jorton: +1 though I'd say log at WARN or INFO for the APR_BAD_DATE 
> > case
> >   rather than "silently" (at normal log-level) dropping the 
> > parsed header?
> >   [also nit: wrapping a lone ap_log_rerror(,APLOG_X) call in
> >   if (APLOGrX(..) is unnecessary/redundant]
> >
> > Modified:
> >  httpd/httpd/branches/2.4.x/   (props changed)
> >  httpd/httpd/branches/2.4.x/CHANGES
> >  httpd/httpd/branches/2.4.x/STATUS
> >  httpd/httpd/branches/2.4.x/server/util_script.c
> >
> > Propchange: httpd/httpd/branches/2.4.x/
> > --
> >Merged /httpd/httpd/trunk:r1748379
> >
> > Modified: httpd/httpd/branches/2.4.x/CHANGES
> > URL: 
> > http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/CHANGES?rev=1879641=1879640=1879641=diff
> > ==
> > --- httpd/httpd/branches/2.4.x/CHANGES [utf-8] (original)
> > +++ httpd/httpd/branches/2.4.x/CHANGES [utf-8] Wed Jul  8 11:39:12 2020
> > @@ -1,6 +1,10 @@
> >-*- coding: 
> > utf-8 -*-
> >   Changes with Apache 2.4.44
> >
> > +  *) core: Drop an invalid Last-Modified header value coming
> > + from a FCGI/CGI script instead of replacing it with Unix epoch.
> > + [Luca Toscano]
> > +
> > *) Add support for strict content-length parsing through addition of
> >ap_parse_strict_length() [Yann Ylavic]
> >
> >
> > Modified: httpd/httpd/branches/2.4.x/STATUS
> > URL: 
> > http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1879641=1879640=1879641=diff
> > ==
> > --- httpd/httpd/branches/2.4.x/STATUS (original)
> > +++ httpd/httpd/branches/2.4.x/STATUS Wed Jul  8 11:39:12 2020
> > @@ -135,31 +135,6 @@ RELEASE SHOWSTOPPERS:
> >   PATCHES ACCEPTED TO BACKPORT FROM TRUNK:
> > [ start all new proposals below, under PATCHES PROPOSED. ]
> >
> > -  *) core: Drop an invalid Last-Modified header value coming
> > - from a (F)CGI script instead of replacing it with Unix epoch.
> > - Warn the users about Last-Modified header value replacements
> > - and violat

Re: iOS 14 / macOS 11 and HTTP/3 support

[Luca Toscano]

Hi Graham,

On Fri, Jul 3, 2020 at 11:42 PM Graham Leggett  wrote:
>
> I recommend the following:
>
> - Start the work on a branch, but ultimately you want this to be in httpd 
> trunk. No rewrites.
> - Look at what a mod_event+ MPM might need to do to provide the UDP and TCP 
> services you need. No sacred cows, if you need to change something, change it.
> - Look at mod_ssl and mod_http, and ask what a mod_h3 might do to replace 
> them both.
> - Look at how a mod_h3 might create a request (request_rec), and how it might 
> process it. Do you want to insulate people from blocking/crashing code here 
> or higher up? How might the MPM help here?
>

Thanks a lot for the email explaining your insights, a very good
reading. I have some questions in my mind that I'd like to add to the
discussion, hopefully they'll not be completely off track (but surely
not very precise).

1) Should we support, in core, QUIC itself rather than only UDP? IIUC
the current mod_h2 model is that a TCP connection, carrying H2
streams, is handled by mod_h2 with its own set of threads, so possibly
having core assigning one mpm-thread to each QUIC stream if needed
could be more intuitive for people that want to create mod_h3 or
similar (who knows what protocols will be running on top of QUIC in 5y
time).
2) Stefan wrote https://icing.github.io/mod_h2/beams.html, as an
elegant solution to bypass an APR limitation for H2. Should we
consider adding more functionalities to APR in light of the H3 work to
ease the job of whoever will write mod_h3?
3) Finally, and I think this is the most important point, how should
we change our current release process to accommodate all the changes
that we are discussing? Releasing 2.6 could be a possible answer, but
we might need more (maybe 2.8/3.0 etc..). Defining when/how we should
release trunk's code has been proven a challenging decision in the
past :D. From the last discussion a lot of things have changed,
especially the fact that we now have a CI (kudos to Joe for all the
work in improving it BTW) but we should start talking about next steps
very soon in my opinion.

Thanks in advance,

Luca

Re: svn commit: r1879253 - /httpd/httpd/trunk/server/util_script.c

[Luca Toscano]

On Mon, Jun 29, 2020 at 9:12 AM Ruediger Pluem  wrote:
>
>
>
> On 6/27/20 11:11 AM, elu...@apache.org wrote:
> > Author: elukey
> > Date: Sat Jun 27 09:11:32 2020
> > New Revision: 1879253
> >
> > URL: http://svn.apache.org/viewvc?rev=1879253=rev
> > Log:
> > server/util_script.c: tune logging Last-Modified header
> >
> > Follow up after Joe's feedback in STATUS:
> >   - If APR_DATE_BAD is returned for Last-Modified, log it at INFO level
> > (as opposed to trace).
> >   - Remove unnecessary guard for APLOGrtrace1(r).
>
>
> As this is now a APLOG_INFO, can you please add an APLOGNO?
> For further details please have a look at docs/log-message-tags/README

Thanks for the pointer, hopefully fixed with r1879348.

Luca

Re: iOS 14 / macOS 11 and HTTP/3 support

[Luca Toscano]

Hi Graham,

On Sun, Jun 28, 2020 at 12:53 PM Graham Leggett  wrote:
>
> On 27 Jun 2020, at 14:48, Luca Toscano  wrote:
>
> > the challenges are the same one discussed in your previous email
> > thread 
> > (https://lists.apache.org/thread.html/eb086eafbd9309eb1efedac3bf3dcc410a95d06206c97e7ade01c254%40%3Cdev.httpd.apache.org%3E).
> > I think that everybody would love to start working/helping on adding
> > HTTP/3 support but the work to be done is huge, involves invasive
> > changes to the httpd's source code and the current dev resources don't
> > have (rightfully) bandwidth to support the current codebase and plan a
> > major refactoring.
>
> I would be careful with wide reaching statements like this.

This is my opinion, supported by my understanding of the thread above,
that's it :). I am just trying to answer to the community with what I
think are the road blockers, please correct me where I am wrong.

> I’ve been working on identifying and removing blockers in various parts of 
> the httpd subsystems that prevent httpd to be cleanly event driven, and most 
> of those blockers have been removed.
>
> The underlying architecture of httpd is very strong, and would support new 
> protocols without too much trouble.

I don't think that my words implied anything against httpd, but in
case you feel in this way apologies. Even if httpd's source is very
strong and has been proven reliable during the years, it was visible
from the development of HTTP/2 that in order to support new protocols,
httpd would have needed to evolve in a significant way, and what I am
trying to say is that it needs resources and people time.

> The main point is that it must be done carefully and properly, but this is 
> not a reason to not do it at all.

Never said (I think) that it is not doable, only that there is a lot
of work to be done.

Luca

Re: iOS 14 / macOS 11 and HTTP/3 support

[Luca Toscano]

Hi,

On Sun, Jun 28, 2020 at 7:58 AM Helmut K. C. Tessarek
 wrote:
>
> On 2020-06-27 08:48, Luca Toscano wrote:
> > the challenges are the same one discussed in your previous email
> > thread
>
> These are all valid points, although in the end absolutely irrelevant.
>
> It's very easy and it doesn't take a genius to figure this out: as soon as
> there is a web server out there that can do HTTP/3, people will just use that
> one instead. All the excuses (even though I do understand how valid they are)
> won't change a thing.

there are already web servers doing HTTP/3, but the httpd project is
not in any "race" to get first to new features, this is not the way
Apache works. We will release HTTP/3 support when the development
community will be ready to deliver a strong and solid codebase, not
before. Supporting HTTP/3 is also a strong requirement that holds for
some use cases (for example, services that are directly Internet
facing), but does not for others (in case of HTTP/2, I am not aware of
a lot of people preferring it to HTTP when dealing with proxied
requests between services etc..).

Again, we welcome any contribution to reach the HTTP/3 goal, as stated
multiple times, since we consider it very important. On our side we'll
also try to do everything to keep supporting httpd and adding new
features in a timely manner (the former is not a simple thing to do
too, given the amount of users to support).

Luca

Re: mod_proxy_fcgi bug using CONTENT_LENGTH and Transfer-Encoding chunked

[Luca Toscano]

Hi Oliver,

your request was duly noted, we might get somebody to look at this
during the next days/weeks, but we can't promise anything. The bug is
sadly not as simple as adding a line of code (see comments in bz
57087), plus the current dev resources have limited bandwidth.

Thanks for following up,

Luca

On Mon, Jun 22, 2020 at 3:00 PM Oliver Dunk  wrote:
>
> Just wanted to resurface this one. I posted it last week, but there hasn’t 
> been any activity yet. I’d still love someone to take a look if they have a 
> minute.
>
> Kind regards,
> Oliver
>
> On 16 Jun 2020, at 22:57, Oliver Dunk  wrote:
>
> Hi,
>
> I wanted to politely ask if anyone could take a look at 
> https://bz.apache.org/bugzilla/show_bug.cgi?id=57087. This is an issue where 
> CONTENT_LENGTH isn’t forwarded correctly when using Transfer-Encoding 
> chunked. I feel uncomfortable bumping an issue in this mailing list, but it’s 
> been open since 2014, and I’m not sure where else to ask.
>
> In the real world, this is causing issues for me when uploading to a 
> Nextcloud instance through macOS finder. Files are uploaded with a size of 0b 
> and macOS shows an error. There’s an issue here which shows some instances of 
> people running in to it: 
> https://github.com/nextcloud/nextcloud-snap/issues/365.
>
> For reference, the same issue was fixed in mod_fcgid a while back: 
> https://bz.apache.org/bugzilla/show_bug.cgi?id=53332 and 
> https://bz.apache.org/bugzilla/show_bug.cgi?id=50274
>
> It would be great if anyone could take a look at this, and see if it is 
> feasible to fix in the near future. The only alternative I can think of is 
> moving my installation away from mod_fcgid.
>
> As a final note, I did consider the user mailing list, but I get the 
> impression few developers look there. Hoping I made the right call.
>
> Kind regards,
> Oliver
>
>

Re: iOS 14 / macOS 11 and HTTP/3 support

[Luca Toscano]

Hi Alex,

the challenges are the same one discussed in your previous email
thread 
(https://lists.apache.org/thread.html/eb086eafbd9309eb1efedac3bf3dcc410a95d06206c97e7ade01c254%40%3Cdev.httpd.apache.org%3E).
I think that everybody would love to start working/helping on adding
HTTP/3 support but the work to be done is huge, involves invasive
changes to the httpd's source code and the current dev resources don't
have (rightfully) bandwidth to support the current codebase and plan a
major refactoring.

Having said that: if anybody is interested in helping, the project
would really be grateful (new contributors are always welcome!).

Luca

On Mon, Jun 22, 2020 at 10:47 PM Alex Hautequest  wrote:
>
> From Apple’s developer beta release notes, the newest Apple code is now 
> shipping with HTTP/3 support. Disabled by default, but can be enabled by 
> users. As of today, HTTP/3 Draft 29 isn’t yet supported.
>
> Alex

Re: Trunk DEFAULT_REL_STATEDIR undeclared

[Luca Toscano]

On Thu, May 7, 2020 at 1:42 PM Joe Orton  wrote:
>
> On Thu, May 07, 2020 at 12:55:03PM +0200, Steffen wrote:
> > Tried to build trunk again after 2 years :)
> >
> > server\core.c(5618,58): error C2065: 'DEFAULT_REL_STATEDIR': undeclared
> > identifier
>
> That was added in r1842929 (October 2018) and nobody has noticed Windows
> was broken before now?  "Discontinued Integration".  Try r1877471 but if
> Windows MPMs use privilege separation like Unix then the STATEDIR should
> be a different directory to logs/ so that's not ideal.
>
> Can someone who cares about Windows pretty plaseee work out how to
> get it building in Travis (or appveyor, or whatever)?

As far as I understood Mario Brandt should be able to help in the near
future, I tried at the time to add it but it is not an easy task
without some knowledge of Window building beforehand :(

Luca

Re: svn commit: r1877345 - in /httpd/httpd/trunk/modules/ssl: ssl_engine_config.c ssl_engine_init.c ssl_engine_pphrase.c ssl_private.h

[Luca Toscano]

Aside from the technical change (that IIUC it is really nice), I
really love when the code is so well documented. Having comments in
the code and in the commit message is great for whoever is in a
similar situation like mine (namely some knowledge of httpd internals
but still very far from Yann/Eric/Ruediger/Joe/Stefan/etc..'s level
:D). So all the extra time put into documentation is really helpful!

Luca


On Mon, May 4, 2020 at 10:37 AM ste...@eissing.org  wrote:
>
> Neat.
>
> > Am 04.05.2020 um 10:32 schrieb jor...@apache.org:
> >
> > Author: jorton
> > Date: Mon May  4 08:32:23 2020
> > New Revision: 1877345
> >
> > URL: http://svn.apache.org/viewvc?rev=1877345=rev
> > Log:
> > mod_ssl: Use retained data API for storing private keys across reloads.
> > Allocate SSLModConfigRec from pconf rather than the process pool.
> >
> > * modules/ssl/ssl_private.h: Add modssl_retained_data_t structure and
> >  move private key storage here from SSLModConfigRec.  Add retained
> >  pointer to SSLModConfigRec.
> >
> > * modules/ssl/ssl_engine_config.c (ssl_config_global_create): Take
> >  pool argument; allocate SSLModConfigRec from there and
> >  initialize mc->retained.  SSLModConfigRec no longer cached for the
> >  process lifetime.
> >  (ssl_init_Module): Sanity check that sc->mc is correct.
> >  (ssl_init_server_certs): Use private keys from mc->retained.
> >
> > * modules/ssl/ssl_engine_pphrase.c
> >  (privkey_vhost_keyid): Rename from asn1_table_vhost_key and
> >  update to use the retained structure.
> >  (ssl_load_encrypted_pkey): Update for above.
> >
> > * modules/ssl/ssl_engine_init.c (ssl_init_Module): Remove
> >  (apparently) redundant call to ssl_config_global_create and
> >  add debug asserts to validate that is safe.
> >
> > Github: closes #119
> >
> > Modified:
> >httpd/httpd/trunk/modules/ssl/ssl_engine_config.c
> >httpd/httpd/trunk/modules/ssl/ssl_engine_init.c
> >httpd/httpd/trunk/modules/ssl/ssl_engine_pphrase.c
> >httpd/httpd/trunk/modules/ssl/ssl_private.h
> >
> > Modified: httpd/httpd/trunk/modules/ssl/ssl_engine_config.c
> > URL: 
> > http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/ssl/ssl_engine_config.c?rev=1877345=1877344=1877345=diff
> > ==
> > --- httpd/httpd/trunk/modules/ssl/ssl_engine_config.c (original)
> > +++ httpd/httpd/trunk/modules/ssl/ssl_engine_config.c Mon May  4 08:32:23 
> > 2020
> > @@ -40,17 +40,17 @@
> > **  _
> > */
> >
> > -#define SSL_MOD_CONFIG_KEY "ssl_module"
> >
> > -SSLModConfigRec *ssl_config_global_create(server_rec *s)
> > +static SSLModConfigRec *ssl_config_global_create(apr_pool_t *pool, 
> > server_rec *s)
> > {
> > -apr_pool_t *pool = s->process->pool;
> > SSLModConfigRec *mc;
> > -void *vmc;
> >
> > -apr_pool_userdata_get(, SSL_MOD_CONFIG_KEY, pool);
> > -if (vmc) {
> > -return vmc; /* reused for lifetime of the server */
> > +if (ap_server_conf && s != ap_server_conf) {
> > +SSLSrvConfigRec *sc = mySrvConfig(ap_server_conf);
> > +
> > +AP_DEBUG_ASSERT(sc->mc);
> > +
> > +return sc->mc;
> > }
> >
> > /*
> > @@ -68,8 +68,6 @@ SSLModConfigRec *ssl_config_global_creat
> > mc->pMutex = NULL;
> > mc->aRandSeed  = apr_array_make(pool, 4,
> > sizeof(ssl_randseed_t));
> > -mc->tVHostKeys = apr_hash_make(pool);
> > -mc->tPrivateKey= apr_hash_make(pool);
> > #if defined(HAVE_OPENSSL_ENGINE_H) && defined(HAVE_ENGINE_INIT)
> > mc->szCryptoDevice = NULL;
> > #endif
> > @@ -86,9 +84,15 @@ SSLModConfigRec *ssl_config_global_creat
> > mc->fips = UNSET;
> > #endif
> >
> > -apr_pool_userdata_set(mc, SSL_MOD_CONFIG_KEY,
> > -  apr_pool_cleanup_null,
> > -  pool);
> > +mc->retained = ap_retained_data_get(MODSSL_RETAINED_KEY);
> > +if (!mc->retained) {
> > +/* Allocate the retained data; the hash table is allocated out
> > + * of the process pool. */
> > +mc->retained = ap_retained_data_create(MODSSL_RETAINED_KEY,
> > +   sizeof *mc->retained);
> > +mc->retained->privkeys = apr_hash_make(s->process->pool);
> > +mc->retained->key_ids = apr_hash_make(s->process->pool);
> > +}
> >
> > return mc;
> > }
> > @@ -248,7 +252,7 @@ void *ssl_config_server_create(apr_pool_
> > {
> > SSLSrvConfigRec *sc = ssl_config_server_new(p);
> >
> > -sc->mc = ssl_config_global_create(s);
> > +sc->mc = ssl_config_global_create(p, s);
> >
> > return sc;
> > }
> >
> > Modified: httpd/httpd/trunk/modules/ssl/ssl_engine_init.c
> > URL: 
> > http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/ssl/ssl_engine_init.c?rev=1877345=1877344=1877345=diff
> > 

Re: Help needed to test Windows builds in Travis

[Luca Toscano]

Hi Mario,

I didn't had the time to do much, so please feel free to take ownership of
the work to do. I am looking forward to see this done properly :)

If I can help please let me know!

Luca

Il mar 28 apr 2020, 21:24 Mario Brandt  ha scritto:

> Hi Luca,
> did you start somewhere yet?
>
> Mario
>
> On Sat, 30 Nov 2019 at 14:56, Luca Toscano  wrote:
> >
> > Hi Mario,
> >
> > will take this as starting point, but since I have zero experience in
> > building anything on Windows it might take a bit to translate
> > everything into a meaningful Travis config. Hope to have something to
> > share during the next days :)
> >
> > Luca
> >
> > Il giorno ven 29 nov 2019 alle ore 05:37 Mario Brandt
> >  ha scritto:
> > >
> > > Hi Luca,
> > >
> > > the cmake post on AL has a batch script, so I think that will do the
> > > job pretty well. https://www.apachelounge.com/viewtopic.php?t=6462
> > >
> > > For a command line build without cmake maybe Gregg or Steffen can help
> > > out more than I can.
> > >
> > > Mario
> > >
> > > On Thu, 28 Nov 2019 at 07:51, Luca Toscano 
> wrote:
> > > >
> > > > Hi Mario,
> > > >
> > > > first of all apologies for the late reply to you, Michal and William,
> > > > I wanted to work on it during the past days but got busy at work. I
> > > > have read some docs but didn't try anything, if you have any
> > > > prototype/suggestion/etc.. I'll be happy to follow up!
> > > >
> > > > Luca
> > > >
> > > > Il giorno mer 27 nov 2019 alle ore 23:48 Mario Brandt
> > > >  ha scritto:
> > > > >
> > > > > Hi Luca,
> > > > > is there any progress on this?
> > > > >
> > > > > Cheers
> > > > > Mario
> > > > >
> > > > > Luca Toscano  schrieb am Do., 7. Nov.
> 2019, 22:56:
> > > > >>
> > > > >> Hi everybody,
> > > > >>
> > > > >> if you build httpd for Windows can you give us a list of commands
> and
> > > > >> steps that you usually do? I am not familiar enough with the
> platform
> > > > >> and [1] is still a bit cryptic to me :)
> > > > >> The end goal is to add the build steps to our Travis config, to
> run
> > > > >> them every time that a commit happens.
> > > > >>
> > > > >> Thanks in advance!
> > > > >>
> > > > >> Luca
> > > > >>
> > > > >> [1] https://httpd.apache.org/docs/2.4/platform/win_compiling.html
>

Re: Travis notifications for trunk?

[Luca Toscano]

On Thu, Apr 23, 2020 at 1:39 PM Stefan Eissing
 wrote:
>
> > Am 23.04.2020 um 12:23 schrieb Ruediger Pluem :
> >
> >
> >
> > On 4/23/20 12:20 PM, Joe Orton wrote:
> >> In the past two weeks we've had three true negative build failures from
> >> Travis on trunk - i.e. bugs were introduced and CI caught them - and one
> >> false negative (where the gcc 9 build broke because of some change in
> >> the Ubuntu repo).  So I think we're reaching the point where signal is
> >> dominating noise.
> >>
> >> Is everybody OK with sending e-mail notifications for CI failures for
> >> trunk to this list?  It's quite likely we will still see occassional
> >
> > +1.
> >
>
> +1

+1, thanks a lot to all the people that have worked to make Travis
checks better!

Re: [RFC] optional Listen options= argument

[Luca Toscano]

On Tue, Apr 21, 2020 at 3:12 PM Eric Covener  wrote:
>
> On Tue, Apr 21, 2020 at 8:53 AM Joe Orton  wrote:
> >
> > A previous conversation [1] about optionally enabling socket options for
> > Listen seemed to gain consensus around adding an optional argument,
> > rather than multiple alternative Listen-like directives.
> >
> > I've attached a patch based on work by both Jan Kaluza and Lubos
> > Uhliarik which implements this, updated for trunk.  Syntax used is:
> >
> >   Listen [IP-address:]portnumber [protocol] [options=keyword,keyword,...]
> >
> > where options is a comma-separated list of keywords.  In this patch the
> > "reuseport" and "freebind" keywords are supported for APR_SO_REUSEPORT
> > and APR_SO_FREEBIND respectively.  Key/value keywords could be used to
> > allow per-listener backlog, TCP buffer sizes etc, though non-boolean
> > options will require extending ap_listen_rec so that needs care.
> >
> > Opinions?  Is there still consensus that extending Listen like this is a
> > good idea?
> >
> > Regards, Joe
> >
> > [1] 
> > http://mail-archives.apache.org/mod_mbox/httpd-dev/201603.mbox/%3c56dd68e5.2040...@redhat.com%3e
>
>
> LGTM

+1 very nice

Re: Preferred service for patch

[Luca Toscano]

Hi!

On Wed, Apr 8, 2020 at 5:35 PM Stéphane Blondon
 wrote:
>
> Hello,
>
> it's possible to provide patch for httpd via bugzilla [1] or Pull
> Request via github [2].
> What is the favorite place to contribute?
>
> 1: https://bz.apache.org/bugzilla/
> 2: https://github.com/apache/httpd

The repo on github is only a mirror so we'll not able to merge patches
from there, but recently we added integration testing via Travis so
pull requests can be automatically tested on multiple platforms.
Having a pull request in github seems fine, maybe the best path would
be to open a bugzilla task explaining the problem etc.. and
referencing any github pull requests in there (as opposed to attach
the patch directly for example).

Hope it helps!

Luca

Re: Branches 2.4.12-dev fails mod_ssl

[Luca Toscano]

Il giorno ven 21 feb 2020 alle ore 13:36 Ruediger Pluem
 ha scritto:
>
>
>
> On 02/21/2020 11:14 AM, Yann Ylavic wrote:
> > Hi Steffen,
> >
> > On Fri, Feb 21, 2020 at 10:23 AM Steffen  wrote:
> >>
> >> Building revision  1874291 failed in Windows.
> >>
> >> mod_ssl.c(27,10): fatal error C1083: Cannot open include file:
> >> 'ap_config_auto.h': No such file or directory
> >
> > Better if you #include "ap_config.h" instead?
> >
>
> This should work. Care to apply?
> BTW: Is there any possibility to have a Travis job run on Windows to catch 
> these?

There is an open email thread about it, the work to be done is not
very easy in my opinion if you are not an expert of building on
Windows. I didn't have time to dive into it yet, but if anybody wants
to send a pull request or commit something please do! :)

Luca

Re: Time for 2.4.42?

[Luca Toscano]

Hi Jim,

Il giorno mar 11 feb 2020 alle ore 15:13 Jim Jagielski
 ha scritto:
>
> Seems like a good time to propose a release... I can RM if desired.

Daniel proposed the same thing a couple of days ago, I'd suggest to
keep only one thread open and +1 in there :)

Luca

Re: Use of [skip ci] in commit messages to avoid Travis builds

[Luca Toscano]

Hi Mike,

TIL, both are in fact accepted:
https://docs.travis-ci.com/user/customizing-the-build#skipping-a-build

Luca

Il giorno lun 10 feb 2020 alle ore 17:32 Mike Rumph
 ha scritto:
>
> This thread mentions [skip ci] but the Travis issue mentions [ci skip] and 
> [ci-skip].
> Are all of these forms recognized?
>
> On Mon, Feb 10, 2020 at 4:23 AM Eric Covener  wrote:
>>
>> On Mon, Feb 10, 2020 at 1:44 AM Ruediger Pluem  wrote:
>> >
>> >
>> >
>> > On 02/08/2020 12:01 PM, Luca Toscano wrote:
>> > > Hi everybody,
>> > >
>> > > Travis is able to read commit messages and skip the CI workflow if the
>> > > "[skip ci]" magic sequence is added somewhere. I keep forgetting about
>> > > it too, but it would be nice if we could start using it to avoid using
>> > > CI resources/workers when not needed (like docs changes, entries in
>> > > STATUS, etc..). I didn't find a way to instruct Travis to avoid
>> > > triggering a build if only certain file types are committed, so the
>> > > only solution for the moment is to manually add the aforementioned
>> > > sequence :(
>> > >
>> > > It is not a big deal to trigger builds even for docs etc.., but the
>> > > ASF resources/workers are limited (and shared among projects IIUC) so
>> > > I am only suggesting to be good neighbors :) If this is totally insane
>> > > or unacceptable I'll back off and stop vouching for it I promise!
>> > >
>> >
>> > Thanks for pointing this out, but I forget this as well in many cases :-).
>> > The only one here who seems to remember this always is IMHO Joe. Maybe
>> > he has some tips how we can get better here.
>> >
>> > Regards
>> >
>> > Rüdiger
>>
>> Is there anything possible in SVN like a pre-commit hook that can see
>> what changed and append [skip ci] unless some other incantantation is
>> present?
>>
>>
>> --
>> Eric Covener
>> cove...@gmail.com

Re: Use of [skip ci] in commit messages to avoid Travis builds

[Luca Toscano]

Speaking of being good neighbors: just committed a change to a xml
file without [skip ci], so easy to miss :( I'll do my best to find
something to automate this..

Luca

Il giorno sab 8 feb 2020 alle ore 12:01 Luca Toscano
 ha scritto:
>
> Hi everybody,
>
> Travis is able to read commit messages and skip the CI workflow if the
> "[skip ci]" magic sequence is added somewhere. I keep forgetting about
> it too, but it would be nice if we could start using it to avoid using
> CI resources/workers when not needed (like docs changes, entries in
> STATUS, etc..). I didn't find a way to instruct Travis to avoid
> triggering a build if only certain file types are committed, so the
> only solution for the moment is to manually add the aforementioned
> sequence :(
>
> It is not a big deal to trigger builds even for docs etc.., but the
> ASF resources/workers are limited (and shared among projects IIUC) so
> I am only suggesting to be good neighbors :) If this is totally insane
> or unacceptable I'll back off and stop vouching for it I promise!
>
> Luca

Use of [skip ci] in commit messages to avoid Travis builds

[Luca Toscano]

Hi everybody,

Travis is able to read commit messages and skip the CI workflow if the
"[skip ci]" magic sequence is added somewhere. I keep forgetting about
it too, but it would be nice if we could start using it to avoid using
CI resources/workers when not needed (like docs changes, entries in
STATUS, etc..). I didn't find a way to instruct Travis to avoid
triggering a build if only certain file types are committed, so the
only solution for the moment is to manually add the aforementioned
sequence :(

It is not a big deal to trigger builds even for docs etc.., but the
ASF resources/workers are limited (and shared among projects IIUC) so
I am only suggesting to be good neighbors :) If this is totally insane
or unacceptable I'll back off and stop vouching for it I promise!

Luca

Re: mod_systemd question

[Luca Toscano]

Il giorno ven 31 gen 2020 alle ore 00:43 Joe Orton 
ha scritto:
>
> On Fri, Jan 31, 2020 at 08:30:06AM +0100, Ruediger Pluem wrote:
> > On 01/31/2020 04:14 AM, Luca Toscano wrote:
> > > Hi everybody,
> > >
> > > I tested a bit the mod_systemd backport proposal (thanks Joe for
> > > working on it!) and I have some doubts, that might be due to my
> > > limited understanding of systemd. I tried the following unit on Debian
> > > 10 (Buster):
> > >
> > > 
> > > # /etc/systemd/system/apache2.service
> > > [Unit]
> > > Description=The Apache HTTP Server
> > > After=network.target
> > >
> > > [Service]
> > > Type=notify
> > > ExecStart=/usr/local/apache2/bin/httpd -k start
> >
> > What if you add -D FOREGROUND to the above?
>
> That should fix the problem described.  The Fedora httpd.service is
> designed to be used with mod_systemd and looks like this:
>
> https://src.fedoraproject.org/rpms/httpd/blob/master/f/httpd.service
>
> We could add a cut-down version of that without some Fedora-specific
> stuff, but there are some policy choices here about what reload & stop
> do so it is maybe best left to distributions, not sure.  At least I
> could document it better!

I would add a bare minimum unit example in the docs with a warning
about special policy choices, just to give a skeleton to our users to
work on. Will try to add an example in the docs during the next days,
but it shouldn't stop the backport, just voted +1!

Luca

Re: mod_systemd question

[Luca Toscano]

Il giorno gio 30 gen 2020 alle ore 23:30 Ruediger Pluem
 ha scritto:
>
>
>
> On 01/31/2020 04:14 AM, Luca Toscano wrote:
> > Hi everybody,
> >
> > I tested a bit the mod_systemd backport proposal (thanks Joe for
> > working on it!) and I have some doubts, that might be due to my
> > limited understanding of systemd. I tried the following unit on Debian
> > 10 (Buster):
> >
> > 
> > # /etc/systemd/system/apache2.service
> > [Unit]
> > Description=The Apache HTTP Server
> > After=network.target
> >
> > [Service]
> > Type=notify
> > ExecStart=/usr/local/apache2/bin/httpd -k start
>
> What if you add -D FOREGROUND to the above?

Seems working really well now, also together with no NotifyAccess set
(defaulting to "main" that seems a good value).

Will do more testing but so far all good!

Thanks!

Luca

mod_systemd question

[Luca Toscano]

Hi everybody,

I tested a bit the mod_systemd backport proposal (thanks Joe for
working on it!) and I have some doubts, that might be due to my
limited understanding of systemd. I tried the following unit on Debian
10 (Buster):


# /etc/systemd/system/apache2.service
[Unit]
Description=The Apache HTTP Server
After=network.target

[Service]
Type=notify
ExecStart=/usr/local/apache2/bin/httpd -k start
ExecReload=/usr/local/apache2/bin/httpd -k graceful
ExecStop=/usr/local/apache2/bin/httpd -k graceful-stop
PrivateTmp=true

[Install]
WantedBy=multi-user.target


The type is "notify" on purpose, the "forking" one seems to work fine
but IIUC it doesn't care at all the sd_notify() calls. I didn't put
any "PidFile" since IIUC it should be provided by mod_systemd via
notifications. This is what I get when trying to start:

Starting The Apache HTTP Server...
Started The Apache HTTP Server.
apache2.service: Got notification message from PID 12291, but
reception only permitted for main PID 12290
apache2.service: Got notification message from PID 12291, but
reception only permitted for main PID which is currently not known
apache2.service: Got notification message from PID 12291, but
reception only permitted for main PID which is currently not known
apache2.service: Succeeded.

It seems as if the children processes signal systemd, but in theory
from what I can see sd_notify is called only from hooks running in the
main process. If I add NotifyAccess=all (default to "main"), the "Got
notification etc.." goes away but I don't see any httpd process up.

I am probably missing something obvious, so if anybody can clarify I'd
be happy :) (vote + documentation will follow in exchange!)

Luca

Re: arm64 support for Travis CI testing

[Luca Toscano]

Il giorno gio 9 gen 2020 alle ore 14:19 Joe Orton 
ha scritto:
>
> The caching does work on arm64 so the build & test only takes 11 minutes
> which is reasonable.
>
> https://travis-ci.org/apache/httpd/jobs/634661216
>
> Looks like this one is the next unreliable test to attack -
>
> 2425# Failed test 17 in t/apache/expr_string.t at line 87
> 2426# Failed test 20 in t/apache/expr_string.t at line 87 fail #2
> 2427t/apache/expr_string.t ..
> 2428Failed 2/44 subtests
> 2429
>
> Can anybody reliably reproduce that failure?

Does it happen only for ARM or also for other architectures? Never
seen this before..

Re: Errored: apache/httpd#137 (trunk - 2688e47)

[Luca Toscano]

Hi Mads,

It might yes, I included also some info in the commit msg. May I ask why
are you asking this? I like to keep dev@ on track to allow more people to
chime in and comment, but I can stop if this is too spammy.

Luca

Il dom 29 dic 2019, 18:17 Mads Toftum  ha scritto:

> On Sat, Dec 28, 2019 at 09:46:28AM +0100, Luca Toscano wrote:
> > sorry for the travis spam, I hope to have improved the config via
> r1872045.
> >
> Would it perhaps make more sense sending this to the same list as commit
> messages or perhaps a new list?
>
> vh
>
> Mads Toftum
> --
> http://flickr.com/photos/q42/
>

Re: Errored: apache/httpd#137 (trunk - 2688e47)

[Luca Toscano]

Update: it seems an issue on the Travis side (
https://travis-ci.community/t/ppc64le-an-error-occurred-while-generating-the-build-script/6598),
plus I noticed other sporadic failures on different part of the CI config.
I just disabled email notifications to dev@ to avoid spamming the list :)

Luca

Il giorno sab 28 dic 2019 alle ore 09:46 Luca Toscano <
toscano.l...@gmail.com> ha scritto:

> Hi everybody,
>
> sorry for the travis spam, I hope to have improved the config via r1872045.
>
> In this last build only one job failed:
> https://travis-ci.org/apache/httpd/jobs/630258333
> The error log says a very informative "An error occurred while generating
> the build script." and nothing more. Since we didn't change ppc64le's
> config in a while I suspect it might be a temporary issue on Travis side.
>
> Luca
>
> Il giorno sab 28 dic 2019 alle ore 09:41 Travis CI 
> ha scritto:
>
>> apache
>>
>> /
>>
>> httpd
>>
>> <https://travis-ci.org/apache/httpd?utm_medium=notification_source=email>
>>
>> [image: branch icon]trunk <https://github.com/apache/httpd/tree/trunk>
>> [image: build has errored]
>> Build #137 has errored
>> <https://travis-ci.org/apache/httpd/builds/630258330?utm_medium=notification_source=email>
>> [image: arrow to build time]
>> [image: clock icon]11 mins and 45 secs
>>
>> [image: Luca Toscano avatar]Luca Toscano
>> 2688e47 CHANGESET →
>> <https://github.com/apache/httpd/compare/c7d7b6d1a3b3...2688e472ff45>
>>
>> test/travis_before_linux.sh: make for loop more resilient
>>
>> This is a follow up to my last commit to this file, to make
>> the for loop more resilient with the following:
>> - use --force in svn export, otherwise the second attempt will
>> always fail due to the dest directory already present.
>> - exit 1 in case the 5 tries end up in a non zero exit code
>> (to fail fast the build).
>>
>>
>>
>> git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1872045
>> 13f79535-47bb-0310-9956-ffa450edef68
>>
>> Want to know about upcoming build environment updates?
>>
>> Would you like to stay up-to-date with the upcoming Travis CI build
>> environment updates? We set up a mailing list for you!
>> SIGN UP HERE <http://eepurl.com/9OCsP>
>>
>> [image: book icon]
>>
>> Documentation <https://docs.travis-ci.com/> about Travis CI
>> Have any questions? We're here to help. 
>> Unsubscribe
>> <https://travis-ci.org/account/preferences/unsubscribe?repository=69847_medium=notification_source=email>
>> from build emails from the apache/httpd repository.
>> To unsubscribe from *all* build emails, please update your settings
>> <https://travis-ci.org/account/preferences/unsubscribe?utm_medium=notification_source=email>.
>>
>> [image: black and white travis ci logo] <https://travis-ci.com>
>>
>> Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF/CEO: Randy
>> Jacops | Contact: cont...@travis-ci.com | Amtsgericht Charlottenburg,
>> Berlin, HRB 140133 B | Umsatzsteuer-ID gemäß §27 a Umsatzsteuergesetz:
>> DE282002648
>>
>

Re: Errored: apache/httpd#137 (trunk - 2688e47)

[Luca Toscano]

Hi everybody,

sorry for the travis spam, I hope to have improved the config via r1872045.

In this last build only one job failed:
https://travis-ci.org/apache/httpd/jobs/630258333
The error log says a very informative "An error occurred while generating
the build script." and nothing more. Since we didn't change ppc64le's
config in a while I suspect it might be a temporary issue on Travis side.

Luca

Il giorno sab 28 dic 2019 alle ore 09:41 Travis CI 
ha scritto:

> apache
>
> /
>
> httpd
>
> <https://travis-ci.org/apache/httpd?utm_medium=notification_source=email>
>
> [image: branch icon]trunk <https://github.com/apache/httpd/tree/trunk>
> [image: build has errored]
> Build #137 has errored
> <https://travis-ci.org/apache/httpd/builds/630258330?utm_medium=notification_source=email>
> [image: arrow to build time]
> [image: clock icon]11 mins and 45 secs
>
> [image: Luca Toscano avatar]Luca Toscano
> 2688e47 CHANGESET →
> <https://github.com/apache/httpd/compare/c7d7b6d1a3b3...2688e472ff45>
>
> test/travis_before_linux.sh: make for loop more resilient
>
> This is a follow up to my last commit to this file, to make
> the for loop more resilient with the following:
> - use --force in svn export, otherwise the second attempt will
> always fail due to the dest directory already present.
> - exit 1 in case the 5 tries end up in a non zero exit code
> (to fail fast the build).
>
>
>
> git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1872045
> 13f79535-47bb-0310-9956-ffa450edef68
>
> Want to know about upcoming build environment updates?
>
> Would you like to stay up-to-date with the upcoming Travis CI build
> environment updates? We set up a mailing list for you!
> SIGN UP HERE <http://eepurl.com/9OCsP>
>
> [image: book icon]
>
> Documentation <https://docs.travis-ci.com/> about Travis CI
> Have any questions? We're here to help. 
> Unsubscribe
> <https://travis-ci.org/account/preferences/unsubscribe?repository=69847_medium=notification_source=email>
> from build emails from the apache/httpd repository.
> To unsubscribe from *all* build emails, please update your settings
> <https://travis-ci.org/account/preferences/unsubscribe?utm_medium=notification_source=email>.
>
> [image: black and white travis ci logo] <https://travis-ci.com>
>
> Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF/CEO: Randy
> Jacops | Contact: cont...@travis-ci.com | Amtsgericht Charlottenburg,
> Berlin, HRB 140133 B | Umsatzsteuer-ID gemäß §27 a Umsatzsteuergesetz:
> DE282002648
>

Re: Errored: apache/httpd#126 (trunk - f190b4e)

[Luca Toscano]

Il giorno sab 21 dic 2019 alle ore 18:21 Luca Toscano <
toscano.l...@gmail.com> ha scritto:

> I can see two failures in here, unrelated to the commit:
>
> 1) svn export -q
> https://svn.apache.org/repos/asf/httpd/test/framework/trunk
> test/perl-framework seems hanging, and I don't see any issue on
> status.apache.org:
>
> +test -v SKIP_TESTING
> +svn export -q https://svn.apache.org/repos/asf/httpd/test/framework/trunk
> test/perl-framework
> No output has been received in the last 10m0s, this potentially indicates
> a stalled build or something wrong with the build itself.
>
> I am wondering if something like the following could help:
>
> for i in {1..5}
> do
> timeout 60 svn export -q
> https://svn.apache.org/repos/asf/httpd/test/framework/trunk
> test/perl-framework
> if [ $? -eq 0 ]; break; else sleep 120; fi
> done
>
> I may have not get the problem though, please add your thoughts!
>

Committed in r1871907 and r1871908


>
> 2) apt install seems failing for network issues, I'd add
> https://docs.travis-ci.com/user/common-build-problems/#travis_retry to
> all our apt configs if you like the idea.
>

I didn't check carefully our config, to use this we should probably avoid
the apt add-on and use apt-get install in the before_install step, since
from the docs (
https://docs.travis-ci.com/user/installing-dependencies/#adding-apt-packages)
it seems that if apt fails there is no retry and the build is marked as
failed.

Joe any opinion?

Happy holidays to everybody :)

Luca

>

Re: Errored: apache/httpd#126 (trunk - f190b4e)

[Luca Toscano]

I can see two failures in here, unrelated to the commit:

1) svn export -q https://svn.apache.org/repos/asf/httpd/test/framework/trunk
test/perl-framework seems hanging, and I don't see any issue on
status.apache.org:

+test -v SKIP_TESTING
+svn export -q https://svn.apache.org/repos/asf/httpd/test/framework/trunk
test/perl-framework
No output has been received in the last 10m0s, this potentially indicates a
stalled build or something wrong with the build itself.

I am wondering if something like the following could help:

for i in {1..5}
do
timeout 60 svn export -q
https://svn.apache.org/repos/asf/httpd/test/framework/trunk
test/perl-framework
if [ $? -eq 0 ]; break; else sleep 120; fi
done

I may have not get the problem though, please add your thoughts!

2) apt install seems failing for network issues, I'd add
https://docs.travis-ci.com/user/common-build-problems/#travis_retry to all
our apt configs if you like the idea.

Luca



Il giorno sab 21 dic 2019 alle ore 17:50 Travis CI 
ha scritto:

> apache
>
> /
>
> httpd
>
> 
>
> [image: branch icon]trunk 
> [image: build has errored]
> Build #126 has errored
> 
> [image: arrow to build time]
> [image: clock icon]11 mins and 2 secs
>
> [image: Lucien Gentis avatar]Lucien Gentis
> f190b4e CHANGESET →
> 
>
> fr doc rebuild.
>
>
> git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1871888
> 13f79535-47bb-0310-9956-ffa450edef68
>
> Want to know about upcoming build environment updates?
>
> Would you like to stay up-to-date with the upcoming Travis CI build
> environment updates? We set up a mailing list for you!
> SIGN UP HERE 
>
> [image: book icon]
>
> Documentation  about Travis CI
> Have any questions? We're here to help. 
> Unsubscribe
> 
> from build emails from the apache/httpd repository.
> To unsubscribe from *all* build emails, please update your settings
> .
>
> [image: black and white travis ci logo] 
>
> Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF/CEO: Randy
> Jacops | Contact: cont...@travis-ci.com | Amtsgericht Charlottenburg,
> Berlin, HRB 140133 B | Umsatzsteuer-ID gemäß §27 a Umsatzsteuergesetz:
> DE282002648
>

Re: Errored: apache/httpd#124 (2.4.x - 29129a5)

[Luca Toscano]

Hi!

Seems unrelated to you commit from the build's logs, I think we'd need to
make some changes to the build scripts to make them more tolerant to
temporary network blips or similar.

Luca

Il giorno gio 19 dic 2019 alle ore 15:12 Steffen  ha
scritto:

> Do not know why I get this.
>
> What to do ?
>
> Steffen
>
> On Thursday 19/12/2019 at 14:53, Travis CI wrote:
>
> apache
>
> /
>
> httpd
>
> 
>
> [ Image ] 2.4.x 
> [ Image ]
> Build #124 has errored
> 
> [ Image ]
> [ Image ] 11 mins and 41 secs
>
> [ Image ] Steffen Land
> 29129a5 CHANGESET →
> 
>
> vote mod_http2
>
>
> git-svn-id:
> https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1871824
> 13f79535-47bb-0310-9956-ffa450edef68
>
> Want to know about upcoming build environment updates?
>
> Would you like to stay up-to-date with the upcoming Travis CI build
> environment updates? We set up a mailing list for you!
> SIGN UP HERE 
>
> [ Image ]
>
> Documentation  about Travis CI
> Have any questions? We're here to help. 
> Unsubscribe
> 
> from build emails from the apache/httpd repository.
> To unsubscribe from *all* build emails, please update your settings
> .
>
> [ Image ] 
>
> Travis CI GmbH, Rigaer Str. 8, 10427 Berlin, Germany | GF/CEO: Randy
> Jacops | Contact: cont...@travis-ci.com | Amtsgericht Charlottenburg,
> Berlin, HRB 140133 B | Umsatzsteuer-ID gemäß §27 a Umsatzsteuergesetz:
> DE282002648
>
>
>

Re: Travis CI failures

[Luca Toscano]

Il giorno dom 1 dic 2019 alle ore 12:05 Luca Toscano
 ha scritto:

> Travis seems not able to deliver emails to dev@, I opened a jira to
> Infra: https://issues.apache.org/jira/browse/INFRA-19508

It was suggested to ask a moderator of httpd-dev@ to approve one of
the Travis emails as way to fix this issue. I am not aware of who is a
moderator, but if anybody is I'd ask to follow up when possible :)

Thanks in advance,

Luca

Re: Help needed to test Windows builds in Travis

[Luca Toscano]

Hi William,

I had time to review this work and it seems automating most of the
steps needed. Would you have time/energy to work with me to integrate
it in the Travis config? I can definitely help for the Travis config
side, but I am still failing to find time to gather a bit of knowledge
about Windows builds.

Thanks in advance,

Luca

Il giorno mar 19 nov 2019 alle ore 00:34 William A Rowe Jr
 ha scritto:
>
> The  https://github.com/appsuite/oss-httpd-build/tree/master/mak tree contains
> all the tooling we've used to generate convenience binaries. 
> Makefile.build-win
> should be what you are looking for; we no longer test .dsp/.mak based builds,
> only CMake logic. But testing both while they remain 'supported' is a very 
> wise
> idea.
>
>
>
> On Fri, Nov 8, 2019 at 12:56 AM Luca Toscano  wrote:
>>
>> Hi everybody,
>>
>> if you build httpd for Windows can you give us a list of commands and
>> steps that you usually do? I am not familiar enough with the platform
>> and [1] is still a bit cryptic to me :)
>> The end goal is to add the build steps to our Travis config, to run
>> them every time that a commit happens.
>>
>> Thanks in advance!
>>
>> Luca
>>
>> [1] https://httpd.apache.org/docs/2.4/platform/win_compiling.html

Travis CI failures

[Luca Toscano]

Hi everybody,

Travis seems not able to deliver emails to dev@, I opened a jira to
Infra: https://issues.apache.org/jira/browse/INFRA-19508

I noticed some errors during the last builds, not code-related but
CI-related: (https://travis-ci.org/apache/httpd/builds)

1)

+svn export -q https://svn.apache.org/repos/asf/httpd/test/framework/trunk
test/perl-framework
svn: E205011: Failure occurred processing one or more externals definitions
The command "./test/travis_before_${TRAVIS_OS_NAME}.sh" failed and
exited with 1 during .


I can repro the problem on my laptop as well, and if I remove the -q I can see:


[..]
Atest/perl-framework/build/config.pl
Atest/perl-framework/scripts/fpm.sh
Atest/perl-framework/LICENSE

Fetching external item into 'test/perl-framework/Apache-Test':
svn: warning: W170013: Unable to connect to a repository at URL
'https://svn.apache.org/repos/asf/perl/Apache-Test/trunk'

Exported revision 1870670.
svn: E205011: Failure occurred processing one or more externals definitions


Checking out the Apache-Test/trunk repo directly works, really strange..

2)

++svn info --show-item last-changed-revision
https://svn.apache.org/repos/asf/apr/apr/branches/1.7.x
svn: E170013: Unable to connect to a repository at URL
'https://svn.apache.org/repos/asf/apr/apr/branches/1.7.x'
svn: E000111: Error running context: Connection refused

+local revision=
+test -f /home/travis/root/apr-1.7.x/.revision-is-
+rm -rf /home/travis/root/apr-1.7.x
+test -d /home/travis/root/apr-1.7.x

+svn export -q -r
https://svn.apache.org/repos/asf/apr/apr/branches/1.7.x
/home/travis/build/apr-1.7.x
svn: E205000: Syntax error in revision argument
'https://svn.apache.org/repos/asf/apr/apr/branches/1.7.x'
The command "./test/travis_before_${TRAVIS_OS_NAME}.sh" failed and
exited with 1 during .


This one seems more a temporary connectivity issue, so I am wondering
if we should add a basic and limited retry/backoff logic in CI to make
the external fetch operations more resilient to network glitches.

Joe any opinion?

Luca

Re: Help needed to test Windows builds in Travis

[Luca Toscano]

Hi Mario,

will take this as starting point, but since I have zero experience in
building anything on Windows it might take a bit to translate
everything into a meaningful Travis config. Hope to have something to
share during the next days :)

Luca

Il giorno ven 29 nov 2019 alle ore 05:37 Mario Brandt
 ha scritto:
>
> Hi Luca,
>
> the cmake post on AL has a batch script, so I think that will do the
> job pretty well. https://www.apachelounge.com/viewtopic.php?t=6462
>
> For a command line build without cmake maybe Gregg or Steffen can help
> out more than I can.
>
> Mario
>
> On Thu, 28 Nov 2019 at 07:51, Luca Toscano  wrote:
> >
> > Hi Mario,
> >
> > first of all apologies for the late reply to you, Michal and William,
> > I wanted to work on it during the past days but got busy at work. I
> > have read some docs but didn't try anything, if you have any
> > prototype/suggestion/etc.. I'll be happy to follow up!
> >
> > Luca
> >
> > Il giorno mer 27 nov 2019 alle ore 23:48 Mario Brandt
> >  ha scritto:
> > >
> > > Hi Luca,
> > > is there any progress on this?
> > >
> > > Cheers
> > > Mario
> > >
> > > Luca Toscano  schrieb am Do., 7. Nov. 2019, 22:56:
> > >>
> > >> Hi everybody,
> > >>
> > >> if you build httpd for Windows can you give us a list of commands and
> > >> steps that you usually do? I am not familiar enough with the platform
> > >> and [1] is still a bit cryptic to me :)
> > >> The end goal is to add the build steps to our Travis config, to run
> > >> them every time that a commit happens.
> > >>
> > >> Thanks in advance!
> > >>
> > >> Luca
> > >>
> > >> [1] https://httpd.apache.org/docs/2.4/platform/win_compiling.html

Re: Help needed to test Windows builds in Travis

[Luca Toscano]

Hi Mario,

first of all apologies for the late reply to you, Michal and William,
I wanted to work on it during the past days but got busy at work. I
have read some docs but didn't try anything, if you have any
prototype/suggestion/etc.. I'll be happy to follow up!

Luca

Il giorno mer 27 nov 2019 alle ore 23:48 Mario Brandt
 ha scritto:
>
> Hi Luca,
> is there any progress on this?
>
> Cheers
> Mario
>
> Luca Toscano  schrieb am Do., 7. Nov. 2019, 22:56:
>>
>> Hi everybody,
>>
>> if you build httpd for Windows can you give us a list of commands and
>> steps that you usually do? I am not familiar enough with the platform
>> and [1] is still a bit cryptic to me :)
>> The end goal is to add the build steps to our Travis config, to run
>> them every time that a commit happens.
>>
>> Thanks in advance!
>>
>> Luca
>>
>> [1] https://httpd.apache.org/docs/2.4/platform/win_compiling.html

Re: svn commit: r1869729 - /httpd/httpd/trunk/.travis.yml

[Luca Toscano]

Il giorno mer 13 nov 2019 alle ore 09:50  ha scritto:
>
> Author: jorton
> Date: Wed Nov 13 08:50:25 2019
> New Revision: 1869729
>
> URL: http://svn.apache.org/viewvc?rev=1869729=rev
> Log:
> Test IRC and e-mail notifications.
>
> Modified:
> httpd/httpd/trunk/.travis.yml
>
> Modified: httpd/httpd/trunk/.travis.yml
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/trunk/.travis.yml?rev=1869729=1869728=1869729=diff
> ==
> --- httpd/httpd/trunk/.travis.yml (original)
> +++ httpd/httpd/trunk/.travis.yml Wed Nov 13 08:50:25 2019
> @@ -117,3 +117,8 @@ before_script:
>
>  script:
>- ./test/travis_run_${TRAVIS_OS_NAME}.sh
> +
> +notifications:
> +  irc: "chat.freenode.net#httpd-dev"
> +  email:
> +- jor...@apache.org
>
>

>From my IRC client (#httpd-dev):

:: Join: travis-ci
(~travis...@ec2-54-89-81-231.compute-1.amazonaws.com) to #httpd-dev
 apache/httpd#44 (trunk - a66fab5 : Joe Orton): The build passed.
 Change view :
https://github.com/apache/httpd/compare/bcfd0565d4fb...a66fab5a0c42
 Build details : https://travis-ci.org/apache/httpd/builds/611268751

Nice!

Luca

Re: trunk APR version requirement

[Luca Toscano]

Il giorno mar 12 nov 2019 alle ore 15:24 Eric Covener
 ha scritto:
>
> >
> > 1. If travis results look stable this week, turn on e-mail notifications
> > to dev@ on Friday.
> >
> > 2. Friday following (22nd), disable the broken bits of buildbot if
> > nobody has salvaged it.
>
> +1

+1

Re: trunk APR version requirement

[Luca Toscano]

Il giorno mar 12 nov 2019 alle ore 12:40 Joe Orton 
ha scritto:
>
> Thanks to everyone who gave feedback, I made the change to require APR
> 1.6 in r1869684.  While I kept the Travis builds passing, buildbot broke
> since it's doing a trunk build against the system APR.  Does anybody
> know if there is a buildbot slave running Bionic?

Or maybe we could think about swapping buildbot with Travis and set
notifications for the latter to dev@. Travis seems working really well
and after all your recent work IIUC it is already testing way more
than what buildbot does :)

Luca

Help needed to test Windows builds in Travis

[Luca Toscano]

Hi everybody,

if you build httpd for Windows can you give us a list of commands and
steps that you usually do? I am not familiar enough with the platform
and [1] is still a bit cryptic to me :)
The end goal is to add the build steps to our Travis config, to run
them every time that a commit happens.

Thanks in advance!

Luca

[1] https://httpd.apache.org/docs/2.4/platform/win_compiling.html

Re: Integration tests running on Docker

[Luca Toscano]

Il giorno gio 7 nov 2019 alle ore 16:49 Yann Ylavic
 ha scritto:
>
> On Thu, Nov 7, 2019 at 4:47 PM Eric Covener  wrote:
> >
> > > @@ -125,6 +125,7 @@
> > >  . non-Unix, single-platform code
> > >  . routine APLOGNO() backports
> > >  . .gdbinit
> > > +. Travis integration: .travis.yml and test/travis*.sh
> >
> > +1
>
> +1

+1 \o/

Re: Integration tests running on Docker

[Luca Toscano]

Il giorno mer 6 nov 2019 alle ore 12:15 Joe Orton 
ha scritto:
>
> On Wed, Nov 06, 2019 at 10:54:56AM +0100, Luca Toscano wrote:
> > I just noticed 
> > https://docs.travis-ci.com/user/installing-dependencies/#installing-dependencies-on-multiple-operating-systems,
> > so the 'addons' will be used by travis only if the os can use them (I
> > was afraid of having windows and apt combined). The only thing that I
> > would add to travis.yml are the ifs to separate
> > before_install/script/etc.. for every os, I think that we should add
> > windows as soon as possible.. Do we have a list of build commands to
> > test in Travis?
>
> I have no idea how to build on anything but Linux :) Someone could do
> osx too, travis supports that...

Me too, I am not a big Windows user, but I am pretty sure that
somebody else in the dev community have some info! Will wait :)

> So it works for PRs and successfully failed for a bogus commit -
>
> https://travis-ci.org/apache/httpd/pull_requests
>
> This one is sensible?
>
> https://github.com/apache/httpd/pull/71/commits/4a46327813e87f3d662e65361e6d9240ce855db5

+1 looks good, we can either keep the ifs in travis config or go for
specific files, I like both approaches!

> I don't know if anything other than Linux can actually be tested like
> this tbh.  (Also trying to context switch between svn & git doing this
> stuff is going to melt my brain!)

We could start with a simple build on Windows, I think it would be
sufficient for the moment.. Osx should be easier, will try to add
something as well. The current format of travis.yaml is simple enough
that people with experience can add their tests, but surely some
documentation is needed!

Luca

Re: Integration tests running on Docker

[Luca Toscano]

Il giorno mar 5 nov 2019 alle ore 10:06 Luca Toscano
 ha scritto:
>
> Il giorno lun 4 nov 2019 alle ore 19:33 Luca Toscano
>  ha scritto:
> >
> > Hi Joe,
> >
> > Il giorno lun 4 nov 2019 alle ore 17:16 Joe Orton 
> > ha scritto:
> > >
> > > Here is a proof of concept -
> > >
> > > https://github.com/notroj/httpd/blob/a73adfc8f1c01fbb6a3d493582f9c49c7c942756/.travis.yml
> > >
> > > This runs using the full test suite using a few different configurations
> > > and also does builds with -Werror and maintainer-mode -
> > >
> > > https://travis-ci.org/notroj/httpd/builds/607213820
> > >
> > > ...this should be very easy to extend with more configurations.
> > >
> > > Can we merge the docker way with this kind of matrix type travis
> > > configuration?
> > >
> >
> > The proof of concept looks great, I wanted to test something similar
> > especially to get timings. It looks like a single test takes from 2 to
> > 5 minutes maximum, that is really impressive, I thought it would have
> > been more. With Docker in theory we could use the os matrix, but the
> > documentation doesn't really suggest any best practice. My idea was to
> > pull different docker images (previously built and uploaded to Docker
> > Hub), and depending on the os in the matrix run the correct docker
> > command in the "script" section (to get an idea
> > https://github.com/apache/arrow/blob/master/.travis.yml, even if they
> > don't use docker).
> >
> > While I research though it might be good to follow your solution and
> > just have a simple Travis file for say ubuntu and windows. Even if it
> > takes 30 mins to complete, it would be a good feedback for anybody
> > committing code to trunk/2.4.x. And it could be possible that this
> > solution is good enough for our use cases, only testing will tell us!
> > So feel free to commit your Travis config to trunk, maybe following
> > the Apache Arrow example? If so others, more experienced with Windows,
> > might be able to chime in and add the missing configuration.
>
> This is my idea:
>
> https://github.com/elukey/httpd/blob/trunk/.travis.yml
>
> https://travis-ci.org/elukey/httpd/builds/607536348
>
> Basically what Joe did, but following what the Apache Arrow Project
> did. There is a little bit of repetition, but in theory with this
> config people are free to add tests for other OSes (osx, windows) and
> I will probably be able to add custom ones with Docker to see the
> difference in execution timings etc.. (Apache Arrow's test do use
> docker, I didn't see it a first).
>

I just noticed 
https://docs.travis-ci.com/user/installing-dependencies/#installing-dependencies-on-multiple-operating-systems,
so the 'addons' will be used by travis only if the os can use them (I
was afraid of having windows and apt combined). The only thing that I
would add to travis.yml are the ifs to separate
before_install/script/etc.. for every os, I think that we should add
windows as soon as possible.. Do we have a list of build commands to
test in Travis?

Luca

Re: Integration tests running on Docker

[Luca Toscano]

Il giorno lun 4 nov 2019 alle ore 19:33 Luca Toscano
 ha scritto:
>
> Hi Joe,
>
> Il giorno lun 4 nov 2019 alle ore 17:16 Joe Orton 
> ha scritto:
> >
> > Here is a proof of concept -
> >
> > https://github.com/notroj/httpd/blob/a73adfc8f1c01fbb6a3d493582f9c49c7c942756/.travis.yml
> >
> > This runs using the full test suite using a few different configurations
> > and also does builds with -Werror and maintainer-mode -
> >
> > https://travis-ci.org/notroj/httpd/builds/607213820
> >
> > ...this should be very easy to extend with more configurations.
> >
> > Can we merge the docker way with this kind of matrix type travis
> > configuration?
> >
>
> The proof of concept looks great, I wanted to test something similar
> especially to get timings. It looks like a single test takes from 2 to
> 5 minutes maximum, that is really impressive, I thought it would have
> been more. With Docker in theory we could use the os matrix, but the
> documentation doesn't really suggest any best practice. My idea was to
> pull different docker images (previously built and uploaded to Docker
> Hub), and depending on the os in the matrix run the correct docker
> command in the "script" section (to get an idea
> https://github.com/apache/arrow/blob/master/.travis.yml, even if they
> don't use docker).
>
> While I research though it might be good to follow your solution and
> just have a simple Travis file for say ubuntu and windows. Even if it
> takes 30 mins to complete, it would be a good feedback for anybody
> committing code to trunk/2.4.x. And it could be possible that this
> solution is good enough for our use cases, only testing will tell us!
> So feel free to commit your Travis config to trunk, maybe following
> the Apache Arrow example? If so others, more experienced with Windows,
> might be able to chime in and add the missing configuration.

This is my idea:

https://github.com/elukey/httpd/blob/trunk/.travis.yml

https://travis-ci.org/elukey/httpd/builds/607536348

Basically what Joe did, but following what the Apache Arrow Project
did. There is a little bit of repetition, but in theory with this
config people are free to add tests for other OSes (osx, windows) and
I will probably be able to add custom ones with Docker to see the
difference in execution timings etc.. (Apache Arrow's test do use
docker, I didn't see it a first).

Luca

Re: Integration tests running on Docker

[Luca Toscano]

Hi Joe,

Il giorno lun 4 nov 2019 alle ore 17:16 Joe Orton 
ha scritto:
>
> Here is a proof of concept -
>
> https://github.com/notroj/httpd/blob/a73adfc8f1c01fbb6a3d493582f9c49c7c942756/.travis.yml
>
> This runs using the full test suite using a few different configurations
> and also does builds with -Werror and maintainer-mode -
>
> https://travis-ci.org/notroj/httpd/builds/607213820
>
> ...this should be very easy to extend with more configurations.
>
> Can we merge the docker way with this kind of matrix type travis
> configuration?
>

The proof of concept looks great, I wanted to test something similar
especially to get timings. It looks like a single test takes from 2 to
5 minutes maximum, that is really impressive, I thought it would have
been more. With Docker in theory we could use the os matrix, but the
documentation doesn't really suggest any best practice. My idea was to
pull different docker images (previously built and uploaded to Docker
Hub), and depending on the os in the matrix run the correct docker
command in the "script" section (to get an idea
https://github.com/apache/arrow/blob/master/.travis.yml, even if they
don't use docker).

While I research though it might be good to follow your solution and
just have a simple Travis file for say ubuntu and windows. Even if it
takes 30 mins to complete, it would be a good feedback for anybody
committing code to trunk/2.4.x. And it could be possible that this
solution is good enough for our use cases, only testing will tell us!
So feel free to commit your Travis config to trunk, maybe following
the Apache Arrow example? If so others, more experienced with Windows,
might be able to chime in and add the missing configuration.

Thanks!

Luca

Re: Time for httpd 2.6.x?

[Luca Toscano]

Il giorno dom 3 nov 2019 alle ore 18:52 Jim Jagielski
 ha scritto:
>
>
>
> On Nov 1, 2019, at 11:59 AM, Luca Toscano  wrote:
>
> Il giorno mar 29 ott 2019 alle ore 18:31 Graham Leggett
>  ha scritto:
>
>
> On 29 Oct 2019, at 15:51, Jim Jagielski  wrote:
>
> My only question regards workflow w/ trunk. Right now, I think we all agree 
> that there are codepaths and features in trunk that are not as stable as we 
> would like. Which is fine... trunk is CTR. But we do need some way to vet 
> those changes (ie, we need to "R" all those "C"s). Some will be accepted, 
> others not. Into what branch do those accepted go? And for the things not 
> accepted for eventually inclusion in 2.5/2.6, do they get removed from trunk? 
> Do they stay in trunk?
>
>
> As I recall from v2.4, we branched from trunk, and then we removed anything 
> we deemed “not ready” from the branch. “not ready” had reasonably simple 
> criteria, like “not documented yet”.
>
> So it seems to me that we need to branch trunk into 2.5.x and "clean up" that 
> branch (items 1-5) and leave trunk alone.
>
>
> +1.
>
> Let’s get a 2.5 out there, and let people bash on it.
>
>
> If this was the previous way of doing things and it worked, +1 as
> well. Is there anybody willing to start this process?
>
>
> I don't want to start any process w/o more consensus about going forward... 
> In particular, branching 2.5.x from trunk...

Some steps would need to be taken before creating the new branch
(basically 1. -> 5. in Yann's list), this is why I am advocating to
have one or more people with clear ideas that start the process. It
seems that overall nobody strongly disagrees with branching off trunk
(after tidying it up as necessary), but if needed we can have a quick
vote about a procedure to follow and then start the work. My 2c :)

Luca

Re: Time for httpd 2.6.x?

[Luca Toscano]

Il giorno mar 29 ott 2019 alle ore 18:31 Graham Leggett
 ha scritto:
>
> On 29 Oct 2019, at 15:51, Jim Jagielski  wrote:
>
> > My only question regards workflow w/ trunk. Right now, I think we all agree 
> > that there are codepaths and features in trunk that are not as stable as we 
> > would like. Which is fine... trunk is CTR. But we do need some way to vet 
> > those changes (ie, we need to "R" all those "C"s). Some will be accepted, 
> > others not. Into what branch do those accepted go? And for the things not 
> > accepted for eventually inclusion in 2.5/2.6, do they get removed from 
> > trunk? Do they stay in trunk?
>
> As I recall from v2.4, we branched from trunk, and then we removed anything 
> we deemed “not ready” from the branch. “not ready” had reasonably simple 
> criteria, like “not documented yet”.
>
> > So it seems to me that we need to branch trunk into 2.5.x and "clean up" 
> > that branch (items 1-5) and leave trunk alone.
>
> +1.
>
> Let’s get a 2.5 out there, and let people bash on it.

If this was the previous way of doing things and it worked, +1 as
well. Is there anybody willing to start this process?

Luca

Re: Integration tests running on Docker

[Luca Toscano]

Hi Daniel,

thanks for following up!

Il giorno mer 30 ott 2019 alle ore 13:05 Daniel Ruggeri
 ha scritto:
>
>
>
> On October 29, 2019 8:12:25 AM CDT, Luca Toscano  
> wrote:
> >Il giorno mar 29 ott 2019 alle ore 12:23 Joe Orton 
> >ha scritto:
> >>
> >> On Sun, Oct 27, 2019 at 06:42:58PM +0100, Luca Toscano wrote:
> >> > Some updates:
> >> >
> >> > - We have support for httpd in travis -
> >https://travis-ci.org/apache/httpd
> >>
> >> Nice, thanks Luca & infra!
> >>
> >> > - In order to configure automatic builds, a travis.yaml file is
> >needed
> >> > in the base path of the repository, in every branch that needs to
> >be
> >> > tested IIUC.
> >> > - My idea is currently to add in trunk a travis.yaml initial
> >> > configuration file, that builds a Docker image like [1] and runs
> >the
> >> > perl test suite.
> >> >
> >> > - Building the Docker image in [1] takes quite a while now (between
> >9
> >> > and 13 mins on my laptop, depending also on network bandwidth
> >etc..)
> >> > so it will need more time before it could be as fast as we need,
> >but
> >> > we have to start from somewhere :)
> >> > - In the travis.yaml file we'd need to put config options about
> >what
> >> > Docker image to build and with what parameters. Ideally I'd like to
> >> > store the Docker images in the httpd-framework repository, and
> >>
> >> Do you mean Dockerfiles rather than images here?
> >
> >Correct yes, my bad!
> >
> >>
> >> > reference them in the Travis config of the httpd branches, but not
> >> > sure if it will be possible. Worst case scenario we'll need to add
> >the
> >> > Docker image in each httpd branch that we want to test (possibly in
> >a
> >> > dedicated dir, like "docker" or "testing".
> >>
> >> I expected two goals for testing:
> >>
> >> 1. configure, build, and test using a variety of different configure
> >> options in the standard Ubuntu environment which Travis provides
> >(enable
> >> different MPMs, different module sets, different
> >> --enable-foo/--with-foo, different gcc versions etc etc)
> >
> >This is also what I was trying to understand in these days, namely if
> >travis could build and test directly without Docker.
> >
> >> 2. configure build and test in a variety of OS environments - this
> >would
> >> make sense using container images.
> >>
> >> I don't have much experience testing inside containers from Travis,
> >but
> >> if we can do both (1) and (2) inside containers that might make
> >sense?
> >> Or can we do both separately?  If you have a .travis.yml which works
> >> then I'd say go commit it and we can work from there.
> >
> >I have currently a Docker file that works for Debian/Ubuntu (and
> >another one for CentOs), and IIUC the same could also be done for
> >other OSes (so having different Docker files for OSX and Win, since
> >Travis seems to support them). The main problem is that it currently
> >takes minutes to build the Docker images, and only after that tests
> >will be able to run. I guess that this problem will be the same if we
> >run directly tests in Travis, but we can give it a try. Another
> >downside about Travis, in my opinion, is that testing locally seems to
> >be more difficult that building Docker images (but I admit my
> >ignorance so it might be different), this is why I preferred Docker in
> >the first place.
>
> This very closely matches my own testing setup. I *really* like this. Thank 
> you for driving this!
>
> >
> >> Or can we do both separately?  If you have a .travis.yml which works
> >> then I'd say go commit it and we can work from there.
> >
> >I currently don't have a working travis yaml, but I'll come up with
> >something during the next days. What I am currently trying to
> >understand is if the Docker files could be stored elsewhere (like in
> >the httpd-framework repo) and downloaded in Travis on demand, to have
> >only one the Travis config in the httpd branches (as opposed to also
> >having Docker files etc..).
> >
> >Let me know your suggestions/thoughts :)
> >
> >Luca
>
> The most I know about Travis is how to spell it, but...
> What if the various Dockerfile-based testing environments were treated as a 
> separate "CI-able" ite

Re: Time for httpd 2.6.x?

[Luca Toscano]

Hi everybody,

Il giorno ven 25 ott 2019 alle ore 12:52 Yann Ylavic
 ha scritto:
>
> So how about:
>  0. github workflow? meanwhile...
>  1. compare trunk/2.4.x (inventory)
>  2. discuss unused/deprecated in trunk to cleanup
>  3. address STALLED entries in trunk if it's not for compatibily reasons
>  4. do API/ABI breaking changes in trunk
>  5. improve trunk w/ things we want in 2.6.x (the real part!)
>  6. trunk => 2.6.x
>  7. remove from 2.6.x things not deprecated in 2. but w/ no champion either
>  8. 2.6.alpha/beta/gamma/rc/whatever
>  9. fix in trunk and backport to 2.6.x
> 10. if not good enough goto 8.
> 11. 2.6.0
> 12+ usual release cycle

The above proposal from Yann seems to be something that could work for
everybody as far as I can tell, what do you think? Do we need an
explicit vote thread?

Luca

Re: Integration tests running on Docker

[Luca Toscano]

Il giorno mar 29 ott 2019 alle ore 12:23 Joe Orton 
ha scritto:
>
> On Sun, Oct 27, 2019 at 06:42:58PM +0100, Luca Toscano wrote:
> > Some updates:
> >
> > - We have support for httpd in travis - https://travis-ci.org/apache/httpd
>
> Nice, thanks Luca & infra!
>
> > - In order to configure automatic builds, a travis.yaml file is needed
> > in the base path of the repository, in every branch that needs to be
> > tested IIUC.
> > - My idea is currently to add in trunk a travis.yaml initial
> > configuration file, that builds a Docker image like [1] and runs the
> > perl test suite.
> >
> > - Building the Docker image in [1] takes quite a while now (between 9
> > and 13 mins on my laptop, depending also on network bandwidth etc..)
> > so it will need more time before it could be as fast as we need, but
> > we have to start from somewhere :)
> > - In the travis.yaml file we'd need to put config options about what
> > Docker image to build and with what parameters. Ideally I'd like to
> > store the Docker images in the httpd-framework repository, and
>
> Do you mean Dockerfiles rather than images here?

Correct yes, my bad!

>
> > reference them in the Travis config of the httpd branches, but not
> > sure if it will be possible. Worst case scenario we'll need to add the
> > Docker image in each httpd branch that we want to test (possibly in a
> > dedicated dir, like "docker" or "testing".
>
> I expected two goals for testing:
>
> 1. configure, build, and test using a variety of different configure
> options in the standard Ubuntu environment which Travis provides (enable
> different MPMs, different module sets, different
> --enable-foo/--with-foo, different gcc versions etc etc)

This is also what I was trying to understand in these days, namely if
travis could build and test directly without Docker.

> 2. configure build and test in a variety of OS environments - this would
> make sense using container images.
>
> I don't have much experience testing inside containers from Travis, but
> if we can do both (1) and (2) inside containers that might make sense?
> Or can we do both separately?  If you have a .travis.yml which works
> then I'd say go commit it and we can work from there.

I have currently a Docker file that works for Debian/Ubuntu (and
another one for CentOs), and IIUC the same could also be done for
other OSes (so having different Docker files for OSX and Win, since
Travis seems to support them). The main problem is that it currently
takes minutes to build the Docker images, and only after that tests
will be able to run. I guess that this problem will be the same if we
run directly tests in Travis, but we can give it a try. Another
downside about Travis, in my opinion, is that testing locally seems to
be more difficult that building Docker images (but I admit my
ignorance so it might be different), this is why I preferred Docker in
the first place.

> Or can we do both separately?  If you have a .travis.yml which works
> then I'd say go commit it and we can work from there.

I currently don't have a working travis yaml, but I'll come up with
something during the next days. What I am currently trying to
understand is if the Docker files could be stored elsewhere (like in
the httpd-framework repo) and downloaded in Travis on demand, to have
only one the Travis config in the httpd branches (as opposed to also
having Docker files etc..).

Let me know your suggestions/thoughts :)

Luca

Re: Integration tests running on Docker

[Luca Toscano]

Some updates:

- We have support for httpd in travis - https://travis-ci.org/apache/httpd
- In order to configure automatic builds, a travis.yaml file is needed
in the base path of the repository, in every branch that needs to be
tested IIUC.
- My idea is currently to add in trunk a travis.yaml initial
configuration file, that builds a Docker image like [1] and runs the
perl test suite.
- Building the Docker image in [1] takes quite a while now (between 9
and 13 mins on my laptop, depending also on network bandwidth etc..)
so it will need more time before it could be as fast as we need, but
we have to start from somewhere :)
- In the travis.yaml file we'd need to put config options about what
Docker image to build and with what parameters. Ideally I'd like to
store the Docker images in the httpd-framework repository, and
reference them in the Travis config of the httpd branches, but not
sure if it will be possible. Worst case scenario we'll need to add the
Docker image in each httpd branch that we want to test (possibly in a
dedicated dir, like "docker" or "testing".

If you are strongly against the above or have more suggestions and
knowledge about Travis please let me know!

Luca


[1] 
https://github.com/elukey/httpd_integration_testing/blob/master/docker/code-testing/debian/Dockerfile

Il giorno mar 22 ott 2019 alle ore 22:04 Luca Toscano
 ha scritto:
>
> Opened an issue to Infra to enable Travis support in github:
> https://issues.apache.org/jira/browse/INFRA-19325
>
> Luca
>
> Il giorno sab 28 set 2019 alle ore 09:39 Luca Toscano
>  ha scritto:
> >
> > Follow up after some work in
> > https://github.com/elukey/httpd_integration_testing:
> >
> > * Use only one Docker image for ubuntu/debian
> > * Created two use cases: code snapshot testing (latest trunk and
> > 2.4.x) vs release candidate testing
> > * Still some issues in installing perl deps on older Debian distro
> > (like Stretch), but the perl suite seems to run fine in both use cases
> > for Debian Buster.
> > * Some issues while running the HTTP/2 test suite, will follow up with 
> > Stefan.
> > * I'll try to add a Docker image for CentOS.
> >
> > Please open pull requests if you have ideas/comments/suggestions/etc.. :)
> >
> > Thanks!
> >
> > Luca
> >
> > Il giorno mer 25 set 2019 alle ore 11:52 Luca Toscano
> >  ha scritto:
> > >
> > > Hi everybody,
> > >
> > > I spent some time reading the previous discussions around the concept
> > > of "CI" and from what I gather, it seems that we didn't reach an
> > > agreement about how to proceed and who is working on it (but I might
> > > be wrong, in case apologies!). From what I can see, there are two
> > > possible working fronts:
> > >
> > > 1) Simplify the usage of the current perl testing suite, adding
> > > docs/tests/etc.. Some people expressed the desire for a more friendly
> > > framework, especially when adding new tests.
> > > 2) Run the testing framework automatically on different environments
> > > to spot anomalies/bugs/etc.. in a timely manner.
> > >
> > > I am a bit ignorant about how to run a proper CI but I created a
> > > little prototype of a Dockerfile able to bootstrap a testing
> > > environment on Debian 10 (Buster) and run the perl test suite:
> > >
> > > https://github.com/elukey/httpd_integration_testing/blob/master/docker/Dockerfile
> > >
> > > The above is only an example, it is missing a lot of things and some
> > > follow up work is needed. But with the following commands, on my
> > > laptop I was able to create a docker image and run the test suite:
> > >
> > > docker build .
> > > docker run $id-of-the-image make check
> > >
> > > Some thoughts:
> > >
> > > 1) The above Dockerfile is really handy since I can easily switch
> > > between Debian versions (Jessie/Stretch/Buster to name the last three)
> > > and run the test suite with different package versions (openssl,
> > > nghttp2, etc..). It should be also easy to create Dockerfiles for
> > > other OS/environments, and run make check in a similar way.
> > > 1-bis) Testing on Windows would still need to be solved, Docker
> > > probably it is not the right solution but we could find something else
> > > to integrate for this specific use case.
> > > 2) Docker also offers a way to open a bash shell in interactive mode,
> > > so it could be easy to run tests on a certain platform when somebody
> > > reports a problem. Or make sure that a new set of tests runs correctly
> &

Re: Time for httpd 2.6.x?

[Luca Toscano]

Thanks a lot for all the answers, waiting for more people to chime in!
What I personally like (at a very high level):

1) create a 2.6.x branch from 2.4.x and start backporting commits from
trunk (with votes etc..), up to the point that we feel 2.6.0 GA is
ready, then follow the regular release process. Christophe's tool for
visualizing diffs between trunk and 2.4.x could be really handy and
helpful.
2) Leave trunk as it is, and consider fixing the issues currently outstanding.
3) Use STATUS to decide what modules/code is to be deprecated.
4) Decide when 2.4.x should be considered EOLed in the future.
5) Decide how/when to bump minor versions in the future, so anybody
adding/contributing code to httpd will have a very high level
expectation of when some code can be released. IIRC during the past
threads it was mentioned that a lot of people contributed with good
code still sitting in trunk due to backporting issues to 2.4.x
(breaking ABI etc..). It is not really fair and it surely drives away
contributors that aim to work on major changes to httpd.

Luca


Il giorno mer 23 ott 2019 alle ore 10:40 Stefan Eissing
 ha scritto:
>
> Hi All,
>
> I agree with CJ here.
>
> As I said in the past, my idea would be to:
> - trunk -> trunk-old,
> - copy 2.4.x -> trunk
> - any feature to bring from trunk-old to the new trunk needs a champion, e.g. 
> someone who does the work (porting and test cases)
>
> To some, that looks like we do not honour past work from people (that was one 
> of the arguments brought forward last time). But it is not only about honour 
> of devs, but also about the sweat and tears of the maintainers during 2.6.x 
> releases. If a feature does not find someone to merge from one branch to 
> another, how could we support this feature in a release? What do the 5-6 
> people who handle 99% of all PRs think about this?
>
> As to the list of things to abandon from 2.4.x, we could handle those like 
> the backport voting in STATUS.
>
> Cheers, Stefan
>
> > Am 23.10.2019 um 10:18 schrieb Christophe JAILLET 
> > :
> >
> > Hi Luco,
> >
> > I've nothing against a 2.6.x branch.
> > My only fear is things in trunk that is incomplete and or not enough 
> > reviewed.
> > In other words, our backport mechanism with at least 3 votes is safeguard 
> > for me.
> >
> > My personal point of view is that 2.6.x should be built with backports from 
> > trunk, just as done actually for 2.4.x.
> > But I know it is not the point of view of many others that consider that 
> > what is in trunk is (or should be) functional, reviewed and tested.
> > Anyway, even if some things are less reviewed, alpha, beta and GA are there 
> > to give others the opportunity to test and report issues, so I is maybe not 
> > that bad to go this way, after all.
> >
> >
> > If we want to go on the 2.6 way, maybe we could open some discussion:
> >
> >   - should we deprecate or remove some 2.4.x functionalities or modules? 
> > (mod_imagemap, mod_cern_meta, maybe NetWare support which has really low 
> > activity, ...)
> >
> >   - should we remove things in trunk that are incomplete or lack consensus: 
> > (this illustrate my fears above)
> > - mod_serf and libserf support? : serf project looks mostly 
> > inactive nowadays, mod_sef have no documentation
> > - pcre2 support? (comments in commits or code say that it is 
> > incomplete and that there is performance or memory management issues)
> > - things listed in 2.4/STATUS: PATCHES/ISSUES THAT ARE STALLED
> >
> >   - should we increase the APR minimum version and cleanup existing code 
> > accordingly? (i.e. switch from some ap_ to apr_ functions)
> >
> >   - we could start to write a "new_features_2_6.html" in order to list new 
> > goodies and incompatibilities and changed behavior
> >
> >
> > just my 2c.
> >
> > CJ
> >
> > Le 23/10/2019 à 08:28, Luca Toscano a écrit :
> >> Not even a comment? :)
> >>
> >> Luca
> >>
> >> Il giorno dom 13 ott 2019 alle ore 20:58 Luca Toscano
> >>  ha scritto:
> >>> Hi everybody,
> >>>
> >>> in trunk's STATUS there are a lot of suggestions about things to
> >>> improve/change for 2.6.x. There have been discussions during the past
> >>> couple of years about how/when/if to create a 2.6 release branch, but
> >>> for a lot of reasons we didn't do any progress. Would it be something
> >>> to consider for the next months?
> >>>
> >>> Luca
> >
> >
>

Re: Time for httpd 2.6.x?

[Luca Toscano]

Not even a comment? :)

Luca

Il giorno dom 13 ott 2019 alle ore 20:58 Luca Toscano
 ha scritto:
>
> Hi everybody,
>
> in trunk's STATUS there are a lot of suggestions about things to
> improve/change for 2.6.x. There have been discussions during the past
> couple of years about how/when/if to create a 2.6 release branch, but
> for a lot of reasons we didn't do any progress. Would it be something
> to consider for the next months?
>
> Luca

Re: Integration tests running on Docker

[Luca Toscano]

Opened an issue to Infra to enable Travis support in github:
https://issues.apache.org/jira/browse/INFRA-19325

Luca

Il giorno sab 28 set 2019 alle ore 09:39 Luca Toscano
 ha scritto:
>
> Follow up after some work in
> https://github.com/elukey/httpd_integration_testing:
>
> * Use only one Docker image for ubuntu/debian
> * Created two use cases: code snapshot testing (latest trunk and
> 2.4.x) vs release candidate testing
> * Still some issues in installing perl deps on older Debian distro
> (like Stretch), but the perl suite seems to run fine in both use cases
> for Debian Buster.
> * Some issues while running the HTTP/2 test suite, will follow up with Stefan.
> * I'll try to add a Docker image for CentOS.
>
> Please open pull requests if you have ideas/comments/suggestions/etc.. :)
>
> Thanks!
>
> Luca
>
> Il giorno mer 25 set 2019 alle ore 11:52 Luca Toscano
>  ha scritto:
> >
> > Hi everybody,
> >
> > I spent some time reading the previous discussions around the concept
> > of "CI" and from what I gather, it seems that we didn't reach an
> > agreement about how to proceed and who is working on it (but I might
> > be wrong, in case apologies!). From what I can see, there are two
> > possible working fronts:
> >
> > 1) Simplify the usage of the current perl testing suite, adding
> > docs/tests/etc.. Some people expressed the desire for a more friendly
> > framework, especially when adding new tests.
> > 2) Run the testing framework automatically on different environments
> > to spot anomalies/bugs/etc.. in a timely manner.
> >
> > I am a bit ignorant about how to run a proper CI but I created a
> > little prototype of a Dockerfile able to bootstrap a testing
> > environment on Debian 10 (Buster) and run the perl test suite:
> >
> > https://github.com/elukey/httpd_integration_testing/blob/master/docker/Dockerfile
> >
> > The above is only an example, it is missing a lot of things and some
> > follow up work is needed. But with the following commands, on my
> > laptop I was able to create a docker image and run the test suite:
> >
> > docker build .
> > docker run $id-of-the-image make check
> >
> > Some thoughts:
> >
> > 1) The above Dockerfile is really handy since I can easily switch
> > between Debian versions (Jessie/Stretch/Buster to name the last three)
> > and run the test suite with different package versions (openssl,
> > nghttp2, etc..). It should be also easy to create Dockerfiles for
> > other OS/environments, and run make check in a similar way.
> > 1-bis) Testing on Windows would still need to be solved, Docker
> > probably it is not the right solution but we could find something else
> > to integrate for this specific use case.
> > 2) Docker also offers a way to open a bash shell in interactive mode,
> > so it could be easy to run tests on a certain platform when somebody
> > reports a problem. Or make sure that a new set of tests runs correctly
> > everywhere.
> > 3) Another use case could be to create a Dockerfile that pulls a
> > specific new release of httpd, installing it and running the test
> > suite on multiple platforms.
> > 4) The same Docker image could also run tests suites like
> > https://github.com/icing/mod_h2/tree/master/test/e2e (that is really
> > nice, I suggest to check it if you haven't done it) to run HTTP/2
> > tests as well.
> > 5) We could even think about having daily docker image builds that
> > take a snapshot of trunk/2.4.x and run the test suites, reporting back
> > any problem.
> >
> > Does the above make sense or it is completely out of scope for our
> > project? In the former case I could spend some time on figuring out
> > how to create what I proposed above, otherwise I'll stop and drop the
> > idea :)
> >
> > Last but not the least, this would be a way to help us testing changes
> > in a more automated way, not a replacement of community based testing
> > and bug reports (stating the obvious for an Apache project but better
> > safe than sorry :).
> >
> > Luca

Re: Migrate to git?

[Luca Toscano]

Hi Joe,

Il giorno lun 14 ott 2019 alle ore 16:27 Joe Orton 
ha scritto:
>
> On Tue, Oct 08, 2019 at 02:17:12PM +0200, Luca Toscano wrote:
> > Il giorno mar 8 ott 2019 alle ore 11:04 Greg Stein 
> > ha scritto:
> > >
> > > Travis CI is possible *today* ... since the svn commits are
> > > replicated over to github, Travis can pick them up and run tests.
> > > Just file an INFRA ticket to enable it.
> > >
> >
> > Thanks for the pointer, will file a task to infra to enable it :)
>
> This would be awesome, did you file something?  If not I can.
>
> Like others I am no fan of git, but so long as we can get the process
> right, I think using PRs with CI would be a significant benefit for the
> project.  At minimum we could avoid some of the trivial build breakage
> type issues which have delayed 2.4 releases recently.
>
> Even for trunk I would like to be able to develop new features and have
> a full test suite run (e.g. w/pool-debug, with different APR releases,
> etc etc) easily available without having to wait an hour with laptop
> fans giving me a headache.
>
> At the moment I think we have a quality control problem for 2.4.x, yet I
> find it hard to justify spending much time on writing test cases because
> that stuff is run so rarely.  How many tests proposed in 2.4.x STATUS
> have had a full test suite run?  I certainly don't always do it.  But if
> we get the tests running all the time automatically it's much easier to
> see a return on investment for improving test coverage.


I still haven't filed the request to infra, I wanted to fix my docker
images first. I have created
https://github.com/elukey/httpd_integration_testing with support for
Debian and Centos (for the moment), to address two use cases:

1) run of the perl/http2 test suites against a new httpd release
2) run of the perl/http2 test suites for the latest version of trunk or 2.4.x

The docker images seem to work fine, but probably there are some stuff
to change/improve. I tried to document myself about Travis and GH, but
still didn't come up with a clear picture. I hope to have something
ready during the next days, but if you have a better
idea/understanding please go ahead and contact infra :)

Thanks!

Luca

Time for httpd 2.6.x?

[Luca Toscano]

Hi everybody,

in trunk's STATUS there are a lot of suggestions about things to
improve/change for 2.6.x. There have been discussions during the past
couple of years about how/when/if to create a 2.6 release branch, but
for a lot of reasons we didn't do any progress. Would it be something
to consider for the next months?

Luca

Re: Migrate to git?

[Luca Toscano]

Il giorno mar 8 ott 2019 alle ore 11:04 Greg Stein 
ha scritto:
>
> Travis CI is possible *today* ... since the svn commits are replicated over 
> to github, Travis can pick them up and run tests. Just file an INFRA ticket 
> to enable it.
>

Thanks for the pointer, will file a task to infra to enable it :)

Luca

Re: Migrate to git?

[Luca Toscano]

Il giorno sab 5 ott 2019 alle ore 22:09 Jim Jagielski
 ha scritto:
>
> Various PMCs have made their default/de-facto SCM git and have seen an 
> increase in contributions and contributors...
>
> Is this something the httpd project should consider? Especially w/ the 
> foundation officially supporting Github, it seems like time to have a 
> discussion about it, especially as we start thinking about the next 25 years 
> of this project :)

+1!

Luca

Re: Cloudflare, Google Chrome, and Firefox Add HTTP/3 Support

[Luca Toscano]

Il giorno sab 28 set 2019 alle ore 00:19 William A Rowe Jr
 ha scritto:
>
> On Fri, Sep 27, 2019 at 10:48 AM Eric Covener  wrote:
>>
>> On Fri, Sep 27, 2019 at 11:20 AM Helmut K. C. Tessarek
>>  wrote:
>> >
>> > On 2019-09-27 03:00, Stefan Eissing wrote:
>> > > I know of no plans to implement HTTP/3 support in Apache httpd.
>> >
>> > I'm sorry, but this seems rather strange to me. So what's the idea behind 
>> > this
>> > decision (or better said the lack of a plan)?
>
>
> It's helpful to understand the nature of the ASF. We are always an incubator
> of great solutions written in code. But there is no ombudsman, no dictates
> which direct projects to do X, Y or Z when it comes to the code that our
> projects create. No demands of implementing features, etc. Everything that
> someone steps up to offer end up right here for discussion on dev@.
>
> There is no planning cabul, or even budget to put this on some coders
> to accomplish. You certainly can add an enhancement request on our
> bugzilla tracker to suggest this, but it is on some motivated party to bring
> the development effort to the table.
>
>>
>> HTTP/3 would be a lot of work, a lot of shoehorning into HTTPD, and
>> likely be de-stabilizing for quite some time.   There is
>> simply nobody (seemingly) working on it.
>
>
>>
>> HTTPD is great and familiar to lots of people, but HTTPD'S age brings
>> a lot of baggage. Lots of other great servers have much
>> less baggage and currently have much more commercial interest and buzz.
>
>
> And it is engineered as an http/1 server. While we can now (with all props
> to Stefan and Tatsuhiro) claim http/2, we don't have the framework to really
> offer great h2 push support and other architecture required for long-lived
> dynamic request pipelines. http/2 and http/3 offer server-originated
> transactions that httpd never anticipated. That would be an entirely new
> module which both mod_http2 and mod_http3 would want to build on,
> and an entirely new definition of the CGI spec and related modules.
>
> So we "speak" http/2 now, and might speak http/3 sooner or later with
> Google's quiche or some other provider. But the server isn't constructed
> to be attentive to both the client's traffic demands and the backend's
> desire to push unsolicited traffic. That would be a fun chasm for some
> coders to jump, and we would welcome them here.

Very nice summary, I'd love to challenge it with a follow up question
(for everybody) - would it be good to start listing some of the
"desired features" somewhere, and find somebody in the community that
could work on them separately? We have a big community and doing the
whole mod_h3 thing might be scary at first sight, meanwhile a more
scoped and challenging project could be appealing. I think that Stefan
did a terrific job in developing mod_h2, we could start from the list
of things that httpd could improve to better support mod_h2. I am
fairly ignorant about the subject but I recall bucket beams
(https://icing.github.io/mod_h2/beams.html) or the fact that mod_h2
has its own set of workers (that may be integrated in mpm-event for
example, IIRC it was discussed in the past). These are only two
examples (that are probably not among the priorities), there are
plenty of things, and they may be the driver to finally push httpd 2.6
(or even more) out if needed.

On the other side, it is true that HTTP/2 and HTTP/3 are often used
only at the border of an infrastructure (reverse proxies etc..) and
HTTP is still probably predominant inside, where there are less
limitations. Knowing that ATS is already working on HTTP/3 is
reassuring on this front, but eventually it would be great that httpd
will do the same. Not saying that httpd is not good as reverse proxy,
just that nowadays light/simple/specialized reverse-proxy + TLS + H/2
have grown a lot in number, and it is easier for them to support new
protocol due to their more recent code history.

Luca

Re: Integration tests running on Docker

[Luca Toscano]

Follow up after some work in
https://github.com/elukey/httpd_integration_testing:

* Use only one Docker image for ubuntu/debian
* Created two use cases: code snapshot testing (latest trunk and
2.4.x) vs release candidate testing
* Still some issues in installing perl deps on older Debian distro
(like Stretch), but the perl suite seems to run fine in both use cases
for Debian Buster.
* Some issues while running the HTTP/2 test suite, will follow up with Stefan.
* I'll try to add a Docker image for CentOS.

Please open pull requests if you have ideas/comments/suggestions/etc.. :)

Thanks!

Luca

Il giorno mer 25 set 2019 alle ore 11:52 Luca Toscano
 ha scritto:
>
> Hi everybody,
>
> I spent some time reading the previous discussions around the concept
> of "CI" and from what I gather, it seems that we didn't reach an
> agreement about how to proceed and who is working on it (but I might
> be wrong, in case apologies!). From what I can see, there are two
> possible working fronts:
>
> 1) Simplify the usage of the current perl testing suite, adding
> docs/tests/etc.. Some people expressed the desire for a more friendly
> framework, especially when adding new tests.
> 2) Run the testing framework automatically on different environments
> to spot anomalies/bugs/etc.. in a timely manner.
>
> I am a bit ignorant about how to run a proper CI but I created a
> little prototype of a Dockerfile able to bootstrap a testing
> environment on Debian 10 (Buster) and run the perl test suite:
>
> https://github.com/elukey/httpd_integration_testing/blob/master/docker/Dockerfile
>
> The above is only an example, it is missing a lot of things and some
> follow up work is needed. But with the following commands, on my
> laptop I was able to create a docker image and run the test suite:
>
> docker build .
> docker run $id-of-the-image make check
>
> Some thoughts:
>
> 1) The above Dockerfile is really handy since I can easily switch
> between Debian versions (Jessie/Stretch/Buster to name the last three)
> and run the test suite with different package versions (openssl,
> nghttp2, etc..). It should be also easy to create Dockerfiles for
> other OS/environments, and run make check in a similar way.
> 1-bis) Testing on Windows would still need to be solved, Docker
> probably it is not the right solution but we could find something else
> to integrate for this specific use case.
> 2) Docker also offers a way to open a bash shell in interactive mode,
> so it could be easy to run tests on a certain platform when somebody
> reports a problem. Or make sure that a new set of tests runs correctly
> everywhere.
> 3) Another use case could be to create a Dockerfile that pulls a
> specific new release of httpd, installing it and running the test
> suite on multiple platforms.
> 4) The same Docker image could also run tests suites like
> https://github.com/icing/mod_h2/tree/master/test/e2e (that is really
> nice, I suggest to check it if you haven't done it) to run HTTP/2
> tests as well.
> 5) We could even think about having daily docker image builds that
> take a snapshot of trunk/2.4.x and run the test suites, reporting back
> any problem.
>
> Does the above make sense or it is completely out of scope for our
> project? In the former case I could spend some time on figuring out
> how to create what I proposed above, otherwise I'll stop and drop the
> idea :)
>
> Last but not the least, this would be a way to help us testing changes
> in a more automated way, not a replacement of community based testing
> and bug reports (stating the obvious for an Apache project but better
> safe than sorry :).
>
> Luca

Re: Integration tests running on Docker

[Luca Toscano]

Hi,

Il giorno mer 25 set 2019 alle ore 13:18 Mads Toftum 
ha scritto:
>
> On Wed, Sep 25, 2019 at 11:52:52AM +0200, Luca Toscano wrote:
> > Does the above make sense or it is completely out of scope for our
> > project? In the former case I could spend some time on figuring out
> > how to create what I proposed above, otherwise I'll stop and drop the
> > idea :)
> >
> To me it seems useful enough to put in the repo with the rest of the
> test code or at least documenting somewhere on http://httpd.apache.org/dev/
> If rolled into httpd test framework, it might make more sense to have a
> make docker (or similar) that spins up an image with the source copied
> in.

Definitely, I started a separate gh project only to show the
prototype, once we have something that works and that we like it might
be better to move it elsewhere.

"make docker" could also be cool to add!

Luca

Re: Integration tests running on Docker

[Luca Toscano]

Hi Daniel,

Il giorno mer 25 set 2019 alle ore 13:24 Daniel Ruggeri
 ha scritto:
>
> I'm a big fan of this idea - it's how I do testing :-)
>
> Unfortunately, I ran into a few issues trying to build and test with Debian 
> maintained dependencies and only ended up with one working test suite: build 
> httpd and all dependencies from sources. But the idea as a whole is sound - 
> we can test against any Linux-based OS distribution which is available in the 
> Docker hub. It could also serve as a nice hint to those that would like to 
> deploy httpd from sources (or even package maintainers) by offering a working 
> recipe that tests clean.
>
> I also raised an inquiry to Infra (but admittedly never followed up) to 
> understand what we'd need to do to have Docker deployed to our CI VM. I 
> understand they use Puppet to manage said VMs so I was hoping to figure out 
> if this would be permissible and then which scripts need to be updated.

Good point about the Debian maintained dependencies, I didn't think
about it. As first step we could simply assume that they don't exist
and build all the deps from source :)
Do you have any special ./configure arg list that you think it would
be worth to use? Or maybe a collection of them, I have seen problems
over time rising from using static vs dso modules, enabling only a
subset of modules, etc.. For the first iteration maybe something that
involves --enable-modules=reallyall is enough?

Thanks!

Luca

Integration tests running on Docker

[Luca Toscano]

Hi everybody,

I spent some time reading the previous discussions around the concept
of "CI" and from what I gather, it seems that we didn't reach an
agreement about how to proceed and who is working on it (but I might
be wrong, in case apologies!). From what I can see, there are two
possible working fronts:

1) Simplify the usage of the current perl testing suite, adding
docs/tests/etc.. Some people expressed the desire for a more friendly
framework, especially when adding new tests.
2) Run the testing framework automatically on different environments
to spot anomalies/bugs/etc.. in a timely manner.

I am a bit ignorant about how to run a proper CI but I created a
little prototype of a Dockerfile able to bootstrap a testing
environment on Debian 10 (Buster) and run the perl test suite:

https://github.com/elukey/httpd_integration_testing/blob/master/docker/Dockerfile

The above is only an example, it is missing a lot of things and some
follow up work is needed. But with the following commands, on my
laptop I was able to create a docker image and run the test suite:

docker build .
docker run $id-of-the-image make check

Some thoughts:

1) The above Dockerfile is really handy since I can easily switch
between Debian versions (Jessie/Stretch/Buster to name the last three)
and run the test suite with different package versions (openssl,
nghttp2, etc..). It should be also easy to create Dockerfiles for
other OS/environments, and run make check in a similar way.
1-bis) Testing on Windows would still need to be solved, Docker
probably it is not the right solution but we could find something else
to integrate for this specific use case.
2) Docker also offers a way to open a bash shell in interactive mode,
so it could be easy to run tests on a certain platform when somebody
reports a problem. Or make sure that a new set of tests runs correctly
everywhere.
3) Another use case could be to create a Dockerfile that pulls a
specific new release of httpd, installing it and running the test
suite on multiple platforms.
4) The same Docker image could also run tests suites like
https://github.com/icing/mod_h2/tree/master/test/e2e (that is really
nice, I suggest to check it if you haven't done it) to run HTTP/2
tests as well.
5) We could even think about having daily docker image builds that
take a snapshot of trunk/2.4.x and run the test suites, reporting back
any problem.

Does the above make sense or it is completely out of scope for our
project? In the former case I could spend some time on figuring out
how to create what I proposed above, otherwise I'll stop and drop the
idea :)

Last but not the least, this would be a way to help us testing changes
in a more automated way, not a replacement of community based testing
and bug reports (stating the obvious for an Apache project but better
safe than sorry :).

Luca

Re: Stale BZ Bug Tracker reports

[Luca Toscano]

Il giorno gio 1 nov 2018 alle ore 22:35 William A Rowe Jr
 ha scritto:
>
> To keep this thread moving (additional feedback is welcomed and 
> appreciated)...

Thanks a lot for this effort William, I really think that having 1700+
bugs opened does not look good for any reporter/user that doesn't know
how we work, since it is easy to get the impression that bugs are
simply opened to take dust for ages (thing that doesn't really
happen).

>
> On Thu, Nov 1, 2018 at 5:03 AM Marion & Christophe JAILLET 
>  wrote:
> >
> > Le 31/10/2018 à 21:52, William A Rowe Jr a écrit :
> > >
> > > There are 715 reports tagged 2.0.0 through 2.3-HEAD of Status NEW or 
> > > NEEDINFO with no Resolution.
> > >
> > > For these bugs I believe we should simply close them with a message that 
> > > this is a mass-update, that the version is beyond EOL, and a request for 
> > > reporters/observers to retest and reopen with the supported version 
> > > number if they are still reproducible using a modern 2.4.x version. But I 
> > > can't determine the best Status/Resolution code... suggestions?
> >
> > +1
> > IMHO, the best status would be CLOSED/WONTFIX. Maybe a new Keyword such as 
> > TooOldAndInactive to ease finding such mass-update?
> > Or ask for a new status TOO_OLD (but I'm not sure it would really be that 
> > useful)
>
> I agree a keyword MassUpdate would be helpful to later identify all tickets 
> closed in any automated way.
>
> WONTFIX doesn't seem to fit; 1) a subset of these have been FIXED,  2) a 
> subset are INVALID, 3) there is a WONTFIX subset (applying to 2.5.x as well.)
>
> I think a new status is right, perhaps RESOLVED/FUTURE is the best course for 
> the mass-change of these defects that are unlikely to be reviewed by the 
> project community in their present state. They are in fact NEEDINFO (we need 
> info that a) there is a bug and b) it still exists), but since we don't have 
> that as a closure state, RESOLVED/FUTURE seems like the best catch-all. Of 
> course this label is no longer endorsed by the bugzilla team, but they don't 
> seem to have substituted anything else; 
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=178923
>
> So I'd read this as the bug needs to be reproduced with a "later" version of 
> httpd, and is subject to reconsideration "later" on further review, but may 
> have already been resolved in a "later" release.

The RESOLVED/FUTURE seems a bit confusing from my point of view, but I
don't really have any good suggestion.. I would personally stick with
something that clearly indicates that the bug was closed due to being
too old.

> > > There are 69 bugs of status REOPENED, and 20 of status ASSIGNED (?). 
> > > These should likely be reviewed by hand and either ACCEPTED against 
> > > 2.4-HEAD, tagged NEEDINFO with a request to re-review (after mass-cleanup 
> > > of NEEDINFO above), or closed as above with an invitation to retest and 
> > > reopen.
> >
> > +1, after by-hand review, as proposed.

+1

> > > There are 255 bugs of Status NEW from 2.4.1-2.4.17, releases which are 
> > > over three years old. For these, the best resolution is probably NEEDINFO.
> >
> > -1.
> > Not sure NEEDINFO is fine for these ones. We should set NEEDINFO after by 
> > hand review, if we NEEDINFO. Or close it if invalid, or leave it as is if 
> > it looks right but no one has looked at it.
> > The reporter has done his job. He has reported what he thinks is enough. WE 
> > should provide an answer or ask for more details.

I agree, even if this effort might be big, sometimes the "history" of
a bug is so long and inconclusive that it is really difficult to judge
what it is the current status. In my opinion we should think about a
quick procedure to keep or close these tasks and apply to all of them.

> You are signing up to reproduce and validate that all of the bugs still exist 
> in the current tree?  I agree that reviewing these 255 bugs would be a noble 
> goal, but who is signing up to the task? Will we simply wait until 2.6.0 has 
> been around for a year or two and then reap all the bugs as described above? 
> I propose we do ask for more details, specifically, that their reported bug 
> still exists, on the presumption it is a bug.
>
> This merits further discussion, and I'm not moving ahead till we've come to 
> some concensus, but we would do well to decide what we are doing here with 
> the group of 3 to 8 year old flavors of 2.4.x.
>
> > > And there are 38 2.4.x NEEDINFO bugs, most of which can likely be closed 
> > > for good as INVALID under a manual review.
> >
> > +1 if older than, let say, 1 year?
> > The number is small, they could also be doubled check by hand. But is is 
> > likely, that it would end as INVALID because the analysis has already been 
> > done, and the reporter does not seem to be interested to answer.

+1

> This number is small, and I am proposing this is a manual effort to either 
> bump the version number perhaps with help from a NEEDINFO request, or closing 
> as 

mod_headers best practices and headers duplicated in the response

[Luca Toscano]

Hi everybody,

apologies if this subject has been brought up in the past but I didn't
find much. I have been working on some bugs like
https://bz.apache.org/bugzilla/show_bug.cgi?id=62380 in which users
report responses with the same header duplicated. To keep the story
short, it seems that everything is due to the fact that sometimes
httpd fills both r->headers_out and r->err_headers_out with the same
header, and both get added to the response.

Example from 62380:

- Simple ProxyPass configuration with mod_proxy_http, the (HTTP)
backend returns a response with the X-Foo header set to baz
- Header always set X-Foo "bar" (in the httpd.conf)

This leads to the following in the response:
[..]
X-Foo: baz
[..]
X-Foo: bar
[..]

This happens because, afaict, mod_proxy_http returns the backend
headers into r->headers_out, while Header always set adds it to
r->err_headers_out. With mod_proxy_fcgi the behavior is different
since both headers end up in r->err_headers_out (so only X-Foo: bar is
present in the response).
I tried to come up with some patch, but the more I think about it the
less I am inclined to change the current behavior, but only to improve
the docs to be more explicit about the two header lists:

Header onsuccess verb [..]  -> works on r->headers_out
Header always verb [..] -> works on r->err_headers_out

So in the above baz/bar example, if the users really wants to be sure
that an override happens, it should:

Header onsuccess unset X-Foo  (onsuccess can be omitted in here, but
it seems clearer to be explicit)
Header always set X-Foo bar

Same thing for all the other verbs of course (setifempty, merge,
etc..). It is also kinda explained in
https://httpd.apache.org/docs/2.4/mod/mod_headers.html#header but in
my opinion not super clearly, it is easy to get confused (like I was)
while reading it the first 3/4 times.

Anything that I am missing or this is "only" a documentation bug fix?
(Also something worth to change to be more user friendly in 2.6 tbh).

Luca

mod_session_cookie and duplicate headers

[Luca Toscano]

Hi everybody,

I noticed that several bugs have been reporting duplicate Set-Cookie
headers in responses when using mod_session_cookie:

https://bz.apache.org/bugzilla/show_bug.cgi?id=56098
https://bz.apache.org/bugzilla/show_bug.cgi?id=55278
https://bz.apache.org/bugzilla/show_bug.cgi?id=60910

And possibly others.. The last one contains a simple patch that avoids
mod-session_cookie to set the header in both r->headers_out and
r->err_headers_out, that IIUC from their semantics should not be used
together for the same header. I tried the patch and it seems working,
but I am not sure if there are drawbacks in committing it. From what I
can see leaving only the header in err_headers_out should not break
any existing use case and fix the duplication.

Thoughts?

Thanks in advance,

Luca

Re: svn commit: r1841620 - /httpd/site/trunk/content/dev/verification.mdtext

[Luca Toscano]

Hi Rainer,

Il giorno sab 22 set 2018 alle ore 05:45 Rainer Jung
 ha scritto:
>
> Hi Luca,
>
> Am 21.09.2018 um 23:47 schrieb Luca Toscano:
> > Hi William,
> >
> > can you write in here the full command to use? Didn't find the -r flag
> > that you mentioned :(
>
> The openssl commandline tool at least since 1.0.2 allows eg.
>
>openssl sha256 -r MYFILE
>
> which outputs the hash file in the same form as eg. "sha256sum -b", so
> allows the hash file to get automatically checked via "sha256sum -c".
>
> Although the "sha256" openssl command is not listen in the help output
> of the openssl binary eg. for 1.0.2, it is available. The full list of
> available digests can be seen in the help output of the openssl dgst
> command:
>
>openssl dgst help
>
> An alternative form of the above command is
>
>openssl dgst -sha256 -r MYFILE
>
> HTH!

It helped indeed, I have (hopefully) improved the verification doc
page in r1841684

Thanks!

Luca

Re: svn commit: r1841620 - /httpd/site/trunk/content/dev/verification.mdtext

[Luca Toscano]

Hi William,

can you write in here the full command to use? Didn't find the -r flag
that you mentioned :(

Thanks!

Luca
Il giorno ven 21 set 2018 alle ore 14:30 William A Rowe Jr
 ha scritto:
>
> You might want to point out the -r flag to OpenSSL, which emits the same 
> output as bintools sha256.
>
>
> On Fri, Sep 21, 2018, 12:30  wrote:
>>
>> Author: elukey
>> Date: Fri Sep 21 17:30:07 2018
>> New Revision: 1841620
>>
>> URL: http://svn.apache.org/viewvc?rev=1841620=rev
>> Log:
>> Remove MD5 traces from documentation and add a SHA256 tutorial.
>>
>> Modified:
>> httpd/site/trunk/content/dev/verification.mdtext
>>
>> Modified: httpd/site/trunk/content/dev/verification.mdtext
>> URL: 
>> http://svn.apache.org/viewvc/httpd/site/trunk/content/dev/verification.mdtext?rev=1841620=1841619=1841620=diff
>> ==
>> --- httpd/site/trunk/content/dev/verification.mdtext (original)
>> +++ httpd/site/trunk/content/dev/verification.mdtext Fri Sep 21 17:30:07 2018
>> @@ -19,10 +19,10 @@ Notice:Licensed to the Apache Softwa
>>  # Verifying Apache HTTP Server Releases
>>
>>  All official releases of code distributed by the Apache HTTP Server Project
>> -are signed by the release manager for the release. PGP signatures and MD5
>> +are signed by the release manager for the release. PGP signatures and SHA
>>  hashes are available along with the distribution.
>>
>> -You should download the PGP signatures and MD5 hashes directly from the
>> +You should download the PGP signatures and SHA hashes directly from the
>>  Apache Software Foundation rather than our mirrors. This is to help ensure
>>  the integrity of the signature files. However, you are encouraged to
>>  download the releases from our mirrors. (Our download page points you at
>> @@ -168,3 +168,23 @@ verifying the signature of a release.
>>  gpg: aka "Jim Jagielski "
>>  gpg: aka "Jim Jagielski "
>>
>> +In order to check the integrity of the downloaded file, you need to 
>> download the source and the related SHA256
>> +hash. For example, assuming a preference for tar.bz, to verify the 2.4.34 
>> release you should end up with two files on disk:
>> +
>> +  * httpd-2.4.34.tar.bz2 (source)
>> +  * httpd-2.4.34.tar.bz2.sha256 (SHA256 hash)
>> +
>> +On most Unix systems then it is only a matter of executing:
>> +
>> +% shasum -a 256 -c httpd-2.4.34.tar.bz2.sha256
>> +httpd-2.4.34.tar.bz2: OK
>> +
>> +Behind the scenes, the command checks that the SHA hash contained in 
>> httpd-2.4.34.tar.bz2.sha256 matches the one
>> +calculated for the file httpd-2.4.34.tar.bz2. The correct result should be 
>> a 'OK' displayed.
>> +
>> +Another way to calculate the SHA256 has for a file is to use openssl:
>> +
>> +% openssl sha -sha256 httpd-2.4.34.tar.bz2
>> +SHA256(httpd-2.4.34.tar.bz2)= 
>> fa53c95631febb08a9de41fd2864cfff815cf62d9306723ab0d4b8d7aa1638f0
>> +
>> +And then verify that the content of httpd-2.4.34.tar.bz2.sha256 matches the 
>> above result.
>> \ No newline at end of file
>>
>>

Re: svn commit: r1840563 - /httpd/httpd/branches/2.4.x/STATUS

[Luca Toscano]

Hi everybody,

Il giorno mar 11 set 2018 alle ore 09:28  ha scritto:
>
> Author: icing
> Date: Tue Sep 11 13:28:19 2018
> New Revision: 1840563
>
> URL: http://svn.apache.org/viewvc?rev=1840563=rev
> Log:
> a cautious vote
>
> Modified:
> httpd/httpd/branches/2.4.x/STATUS
>
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1840563=1840562=1840563=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Tue Sep 11 13:28:19 2018
> @@ -144,6 +144,12 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>   trunk patch: http://svn.apache.org/r1832092
>   2.4.x patch: svn merge -c 1832092 ^/httpd/httpd/trunk .
>   +1: elukey
> + +0.5: icing: as I read this, the change preserves the special status of 
> headers in
> + ->err_headers_out, since swapping the tables makes former error 
> headers normal ones.
> + But it is hard to see of this was ever intentional or not. Lack of 
> regressions
> + in testing may meaning someone out there relies on the former, 
> unverified
> + behaviour. OTOH, it fixes and error you saw and added test for. So, 
> I am cautiously
> + for the change.

Any more suggestions/feedback like the following for this change? I'd
really like to know if it needs more work or if it is something non
backportable to 2.4.x :)

PS: thanks Stefan for the review!

Thanks!

Luca

Re: 2.4.35 in Sept?

[Luca Toscano]

2018-08-29 16:10 GMT+02:00 Jim Jagielski :
> Looking to maybe do an interim release in Sept...
>
> Comments? Feedback?

+1 from me, looking forward to see mod-ratelimit fixed :)

Luca

Re: Bug in mod_ratelimit?

[Luca Toscano]

Hi Cory,

Il giorno gio 2 ago 2018 alle ore 14:10 Yann Ylavic
 ha scritto:
>
> On Fri, Jul 27, 2018 at 6:56 PM, Cory McIntire  wrote:
> >
> > {quote}
> > While it will probably resolve the issues we saw, I’d be hesitant to
> > move forward with that patch as it modifies how all output filters
> > work with HEAD requests, this is too large a change, especially when
> > the bug(s) being addressesed are in a single module.
> >
> > I’d recommend making mod_ratelimit do the same “optimization” hack
> > that other modules for HEAD requests instead, and keep the surface
> > area for this bug fix isolated to mod_ratelimit only.
> >
> > Something like what mod_brotli does:
> >
> >  if (r->header_only && r->bytes_sent) {
> >  ap_remove_output_filter(f);
> >  return ap_pass_brigade(f->next, bb);
> >  }
> >  {quote}
> >
> > If there are any further adjustments to this patch we’d be happy to
> > take a look, just let us know.
>
> Sorry, I missed that proposal.
>
> So, AIUI, the above plus moving the ratelimit filter after the "CHUNK"
> filter, right?
> Otherwise I don't see where we address the "header
> ratelimited/retained before chunking" case (regardless of
> HEAD/GET/..).
> IOW, the patch (limited to mod_ratelimit) would be something like:
>
> @@ -123,6 +123,13 @@ rate_limit_filter(ap_filter_t *f, apr_bucket_briga
>  APR_BRIGADE_PREPEND(bb, ctx->holdingbb);
>  }
>
> +if (f->r->header_only && f->r->bytes_sent) {
> +ap_remove_output_filter(f);
> +rv = ap_pass_brigade(f->next, bb);
> +apr_brigade_cleanup(bb);
> +return rv;
> +}
> +
>  while (!APR_BRIGADE_EMPTY(bb)) {
>  apr_bucket *e;
>
> @@ -327,7 +334,7 @@ static void register_hooks(apr_pool_t *p)
>  ap_register_output_filter(RATE_LIMIT_FILTER_NAME, rate_limit_filter,
> -  NULL, AP_FTYPE_PROTOCOL + 3);
> +  NULL, AP_FTYPE_CONNECTION - 1);
>
> I think it does not work for the case where a HEAD response has a
> large header but "Content-Length: 0", such that rate_limit_filter()
> retains the last buckets but is never called to release them (i.e.
> EOS).
> Really we shouldn't have any (protocol) filter that eats EOS, this is
> the garantee that each request filter sees when it should terminate
> and bail out (that's also the purpose of
> ap_finalize_request_protocol() for instance).
>
> r1837130 looks like the right fix to me.

sorry for the late ping but I was on holidays. I know that you and
your team expressed some doubts about the fix for mod_ratelimit, but
it seems that Yann's fix is the correct way to go for me too. Any
thoughts? It would be great to reach some consensus within the
community before proceeding :)

Thanks!

Luca

Re: automation and test cases

[Luca Toscano]

Hi Stefan,

Il giorno mar 7 ago 2018 alle ore 11:08 Stefan Eissing
 ha scritto:
>
> ...are the main things that will improve quality. Because
> it enables not only you but everyone else to be more productive.
>
> If you feel for this project, work on it.

Answering a bit late to this email due to holidays. I do agree with
you that this is a good goal, and I'd like to participate but IIRC we
didn't go far the last time that we discussed these topics. There were
people expressing a lot of desires and suggestions but I don't recall
if we ever got to any actionable to work on.

Speaking about test cases and the perl framework, some things that
would be great to work on are (imho):
1) define recipes for common task and document them (like, how do I
test a call to a PHP script?). Having cookbooks would encourage people
to add simple and more complex tests, because they wouldn't spend a
ton of time checking the test repo looking for a magical combination
of perl commands to make something work (happens to me all the time).
2) try to reliably run the test framework against 2.4.x's code more
often (and on more platform/configurations) rather than relying on
people manually doing it, to catch issues earlier.

There are surely a ton more things to do, these are my 2c :)

Luca

Re: Bug in mod_ratelimit?

[Luca Toscano]

Hi Cory,

2018-07-20 13:47 GMT+02:00 Yann Ylavic :
> Hi Cory,
>
> On Thu, Jul 19, 2018 at 11:23 PM, Cory McIntire  wrote:
>>
>> We’re going to revert to the 2.4.33 version of mod_ratelimit for now.
>>
>> HEAD requests with large amount of headers were still problematic in our 
>> testing with both versions of the patch applied.
>
> Thanks for letting us know.
>
> I think the right fix is the attached patch (tested with GET/HEAD with
> large header and/or body, seems to work).
> If by any chance you can give it a try...

In the meantime, other people are testing Yann's last patch in
https://bz.apache.org/bugzilla/show_bug.cgi?id=62568 (it is attached
in there). If you could chime in whenever you have time and let us
know your thoughts it would be really great.

Thanks in advance!

Luca

Re: Bug in mod_ratelimit?

[Luca Toscano]

Hi Yann,

2018-07-19 22:55 GMT+02:00 Yann Ylavic :
>
> Hi Luca,
>
> On Thu, Jul 19, 2018 at 9:59 PM, Luca Toscano  wrote:
> >
> > I confirm that it works fine in my local dev environment, plus now gdb's
> > dump_brigade makes sense, but I am not sure that I have understood what was
> > wrong. In my tests, the headers were the first heap bucket reported by
> > dump_brigade in gdb (before your patch), so IIUC we were saving them to
> > holdingbb and then finally passing them via ap_pass_brigade. But I thought
> > that this was they way to go, what am I missing? I am pretty sure this is
> > basic filter knowledge but I have missed it up to now, if not present in the
> > docs already I'll make sure to add it.
>
> You probably didn't test with chunked encoding (neither did I!), see
> how ap_http_header_filter() adds the "CHUNK" filter after the headers
> have been sent, so if anything retains the headers they will be
> considered as the body by the (late) ap_http_chunk_filter()..


Thanks a lot for the explanation, it took me a while to understand
what you meant but I think I kinda got it. You are saying that
ap_http_header_filter specifically passes the brigade containing the
headers to the next filters in the chain, hoping that it would reach
the client before any attempt to insert the chunk filter is made. But
in this case, the rate_limit_filter interferes with
ap_http_header_filter's assumptions, and buffers the brigade
containing the headers (that never reaches the external client). At
this point, the chunk filter is added by ap_http_header_filter, and
during the next ap_pass_brigade call the buffered data is passed by
rate_limit_filter to the chain, eventually reaching the chunk filter,
that will interpret it as body (making the mess that Cory reported).

Is my understanding vaguely resembling what happens? If so I think it
would be an interesting use case to document in
https://httpd.apache.org/docs/trunk/developer/output-filters.html :)

> @team: the issue with HEAD request is that ap_http_header_filter()
> eats the final EOS (along with the body), I don't think this is
> correct because downstream filters may depend on it to bail out (like
> rate_limit_filter() does now).
> So how about the attached patch both with regard to level
> FTYPE_CONNECTION - 1 for ratelimit filter and ap_http_header_filter()
> to forward the final EOS?

It looks good to me, but I'd really like to get others opinion on how
to fix this problem. What do others think about it?

Thanks a lot for this fix!

Luca

Re: Bug in mod_ratelimit?

[Luca Toscano]

Hi Yann,

2018-07-19 21:41 GMT+02:00 Yann Ylavic :

> On Thu, Jul 19, 2018 at 8:23 PM, Luca Toscano 
> wrote:
> >
> > Yann, any idea?
>
> Looks like we missed the simplest case :/
>
> Index: modules/filters/mod_ratelimit.c
> ===
> --- modules/filters/mod_ratelimit.c(revision 1835556)
> +++ modules/filters/mod_ratelimit.c(working copy)
> @@ -208,7 +208,7 @@ rate_limit_filter(ap_filter_t *f, apr_bucket_briga
>  ap_remove_output_filter(f);
>  }
>  else if (!APR_BUCKET_IS_FLUSH(e)) {
> -if (APR_BRIGADE_EMPTY(bb)) {
> +if (ctx->do_sleep && APR_BRIGADE_EMPTY(bb)) {
>  /* Wait for more (or next call) */
>  break;
>  }
> _
>

I confirm that it works fine in my local dev environment, plus now gdb's
dump_brigade makes sense, but I am not sure that I have understood what was
wrong. In my tests, the headers were the first heap bucket reported by
dump_brigade in gdb (before your patch), so IIUC we were saving them to
holdingbb and then finally passing them via ap_pass_brigade. But I thought
that this was they way to go, what am I missing? I am pretty sure this is
basic filter knowledge but I have missed it up to now, if not present in
the docs already I'll make sure to add it.

Other mea culpa: I tested this change with a proxied scenario in which a
file was transferred from a proxied backed to the client via httpd, and I
didn't notice any problem. I also thought that the tests suite would have
caught a case like this one, but I was terribly wrong, so if possible I'll
try to add a test as follow up to make sure that basic proxied responses
are not mangled like this.

Really sorry for this mess, I should have been more careful before
committing. Lesson learned for the next time :(

Luca

Re: Bug in mod_ratelimit?

[Luca Toscano]

Hi again Cory,

2018-07-19 19:02 GMT+02:00 Cory McIntire :

> Hi Luca,
>
> Sorry for quick reply but we were able to replicate it just now:
>
> # setup a brand new install of wp on a domain (don't have to go through
> the 'db' setup process, just configure wp-config.php to get to install.php
> redirect)
> # install mod_ratelimit, and setup a vhost.conf with the ratelimit config
> for the domain
> # restart apache
> # visit site, see you are getting the "redirect" content instead of
> actually being redirected:
>
> •  curl -H'Host: cptestaddon.com' http://10.215.218.12/
> • HTTP/1.1 302 Moved Temporarily
> • Date: Thu, 19 Jul 2018 16:47:07 GMT
> • Server: Apache
> • X-Powered-By: PHP/5.6.36
> • Expires: Wed, 11 Jan 1984 05:00:00 GMT
> • Cache-Control: no-cache, must-revalidate, max-age=0
> • Pragma: no-cache
> • Location: http://cptestaddon.com/wp-admin/install.php
> • Transfer-Encoding: chunked
> • Content-Type: text/html; charset=UTF-8
> • 0
>
> It is any CGI app but WP was an easy target to replicate on.
>
>
I can see the same thing with a simple php script that says "this is a
test" on my testing environment:

vagrant@stretch:~$ curl -k https://localhost/test.php
HTTP/1.1 200 OK
Date: Thu, 19 Jul 2018 18:15:09 GMT
Server: Apache/2.4.34-dev (Unix) OpenSSL/1.1.0f
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

this is a test!
0

(Note the zero at the end)

So this is a bug introduced by the latest patch for sure, but I still have
no idea where it comes from. I apologize for this issue, I was convinced
that the new code was tested but apparently I missed the most basic use
cases.

Yann, any idea?

Luca

Re: Bug in mod_ratelimit?

[Luca Toscano]

Hi Cory,

2018-07-19 16:10 GMT+02:00 Cory McIntire :

> Hello all,
>
> We’re starting to see some issues where mod_ratelimit change here:
>
>   *) mod_ratelimit: fix behavior when proxing content. PR 62362.
>  [Luca Toscano, Yann Ylavic]
>
> Is causing some sites to load in plain text/source code…
>
> We haven’t found the connection beyond unloading mod_ratelimit which
> resolves the issue,
>  and its not happening everywhere, just curious if anyone else is seeing
> this?
>
> I’ll report back once I have more info on further factors involved.
>

Thanks a lot for reporting this. Can you add a bit more info about how to
reproduce (httpd config I mean)? Anything relevant in the error logs?

Luca

Re: [users@httpd] mod_ratelimit working by steps ?

[Luca Toscano]

Hi Yann!

2018-06-18 12:04 GMT+02:00 Yann Ylavic :

> Hi Luca,
>
> On Sat, Jun 16, 2018 at 8:56 AM, Luca Toscano 
> wrote:
> >
> > While reading the code I didn't get if one particular use case may or may
> > not happen at all, it is the only big doubt that I have before
> committing.
> > Is it possible that the first bb to process (during the first invocation
> of
> > the filter) is equal to the chunk_size + ctx->burst? IIUC the code
> relies on
> > the fact that bb is not empty to decide whether or not to pass ctx->tmpp
> to
> > the next filter in the chain, but if this is not the case then it may
> loop?
>
> I'm not sure to follow, a loop in mod_ratelimit's filter itself?
> 1/ If bb is given empty then the filter does nothing (AFAICT), this
> should not happen but it's not each filter's business to handle the
> case either (just not crash or loop indefinitely of course).
> 2/ If bb is less than chunk_size + burst bytes, then data are retained
> in holdingbb until the next call (provided not EOS/FLUSH and so
> on...).
> 3/ If bb is greater or equal to chunk_size + burst bytes (the case you
> are talking about AIUI), then each chunk is sent rate limited and
> either 1/ or 2/ applies for the rest.
>
> Which case exactly do you care about?
>

Thanks for the patience! You are of course correct, I didn't put the use
case that I had in mind into a proper context. Basically I was worried that
this may have happened:

1) brigade with something less or equal than chunk size + burst
2) apr_brigade_partition returning the brigade's sentilel
3) no EOS and bb empty, nothing passed to the next filter
4) data saved in holdingbb
5) next filter invocation, holdingbb moved to bb, back to 1) ...

While writing it down it is of course clear that a loop cannot happen,
because even if 1-4) happens, then there will be another filter invocation
that will either bring EOS or more data, that will make things go forward.

It was a stupid doubt but I understood a lot of things, I'll try to avoid
spamming the mailing list the next time :D

Will commit your patch as soon as possible!

Luca

Re: [users@httpd] mod_ratelimit working by steps ?

[Luca Toscano]

2018-06-08 12:43 GMT+02:00 Luca Toscano :

> Hi Yann,
>
> 2018-06-07 12:59 GMT+02:00 Yann Ylavic :
>
>> On Thu, Jun 7, 2018 at 10:17 AM, Yann Ylavic 
>> wrote:
>> > On Thu, Jun 7, 2018 at 9:21 AM, Yann Ylavic 
>> wrote:
>> >>
>> >> Feel free to take what can serve you, or burn it ;)
>> >
>> > Burn it! Here is v2, hopefully a less buggy :)
>>
>> FWIW, a v3 with less changes and a few comments where needed.
>> Less changes because it keeps using tmpbb in the RATE_LIMIT loop (with
>> APR_BRIGADE_CONCAT(ctx->tmpbb, bb) first).
>> HTH...
>>
>
> All my consistency tests are green, and after a first (and quick!) pass of
> the code in your patch I can definitely say that your version is much
> better and coincise, I like it! Will try to review it in detail during the
> next days and then I'll commit it.
>

Sorry for the delay :)

While reading the code I didn't get if one particular use case may or may
not happen at all, it is the only big doubt that I have before committing.
Is it possible that the first bb to process (during the first invocation of
the filter) is equal to the chunk_size + ctx->burst? IIUC the code relies
on the fact that bb is not empty to decide whether or not to pass ctx->tmpp
to the next filter in the chain, but if this is not the case then it may
loop?

I tried to dump the brigades of my tests (proxying content via
mod_proxy_http and directly from httpd), the response headers in the
beginning seems to always guarantee an initial bb of some bytes that make
everything work.

Not sure if this is a valuable comment, the rest looks super good :)

Thanks for the patience!

Luca

Re: [users@httpd] mod_ratelimit working by steps ?

[Luca Toscano]

Hi Yann,

2018-06-07 12:59 GMT+02:00 Yann Ylavic :

> On Thu, Jun 7, 2018 at 10:17 AM, Yann Ylavic  wrote:
> > On Thu, Jun 7, 2018 at 9:21 AM, Yann Ylavic 
> wrote:
> >>
> >> Feel free to take what can serve you, or burn it ;)
> >
> > Burn it! Here is v2, hopefully a less buggy :)
>
> FWIW, a v3 with less changes and a few comments where needed.
> Less changes because it keeps using tmpbb in the RATE_LIMIT loop (with
> APR_BRIGADE_CONCAT(ctx->tmpbb, bb) first).
> HTH...
>

All my consistency tests are green, and after a first (and quick!) pass of
the code in your patch I can definitely say that your version is much
better and coincise, I like it! Will try to review it in detail during the
next days and then I'll commit it.

Thanks!

Luca

Re: [users@httpd] mod_ratelimit working by steps ?

[Luca Toscano]

2018-06-05 8:19 GMT+02:00 Luca Toscano :

>
>
> 2018-06-04 16:22 GMT+02:00 Luca Toscano :
>
>> Hi Yann!
>>
>> 2018-06-04 15:59 GMT+02:00 Yann Ylavic :
>>
>>> Hi Luca,
>>>
>>> On Mon, Jun 4, 2018 at 11:23 AM, Luca Toscano 
>>> wrote:
>>> >
>>> > To keep archives happy: opened
>>> > https://bz.apache.org/bugzilla/show_bug.cgi?id=62362 and added a
>>> patch in
>>> > there, if anybody wants to review it and give me suggestions I'd be
>>> happy :)
>>>
>>> The semantic of tmpbb is not very clear in your patch, it's both the
>>> brigade where buckets are saved (for the next call?) and the one that
>>> gets passed to the next filter finally.
>>> It doesn't look right to me, if tmpbb is to be forwarded in the same
>>> pass there is no need to change its buckets' lifetime.
>>> Couldn't we ap_save_brigade(f, >holdingbb, >tmpbb, ..) at
>>> the end of the filter only?
>>>
>>
>> Thanks for the review, bare in mind that this is my first "big" patch so
>> I'd probably need a better grasp of the internals first :) I haven't
>> touched holdingbb since I saw that was used elsewhere (in RATE_FULLSPEED),
>> but I can try to check it as well. The idea of my patch (that I aimed to)
>> is to pass the ctx->tmbbb only if it reaches chunk_size (or EOS is seen)
>> and buffer otherwise the buckets in it using ap_save_brigade (waiting for
>> the next call to see if chunk_size is reached).
>>
>> So if I got your point correctly you would use ctx->holdingbb to store
>> the buckets (and changing their lifetime possibly) between calls, and tmpbb
>> only within the same filter invocation?
>>
>>
> After re-reading the code and I think I got your point Yann, I'll try to
> re-work my idea and create a new patch. Thanks!
>

Tried to improve the ctx->tmpbb usage adding a new ctx->buffer brigade,
with the only duty of buffering buckets between filter's invocations (if
needed):

http://home.apache.org/~elukey/httpd-trunk-mod_ratelimit-rate_limit_proxied_content.patch

I didn't touch holdingbb since it is already used to allow buckets to skip
the rate limit and go full speed, even if I am not sure if this
functionality has it ever been used. I would prefer not to touch that
special logic and fix only this use case.

I may have misunderstood Yann's suggestions, if so sorry for the spam, but
I'd be glad to get a bit more info about how a better patch should look
like :)

I promise documentation in the docs/developer section when the task is
done! (I can add a "If even Luca did this, you can as well!" section in the
docs :P)

Luca

Re: [users@httpd] mod_ratelimit working by steps ?

[Luca Toscano]

2018-06-04 16:22 GMT+02:00 Luca Toscano :

> Hi Yann!
>
> 2018-06-04 15:59 GMT+02:00 Yann Ylavic :
>
>> Hi Luca,
>>
>> On Mon, Jun 4, 2018 at 11:23 AM, Luca Toscano 
>> wrote:
>> >
>> > To keep archives happy: opened
>> > https://bz.apache.org/bugzilla/show_bug.cgi?id=62362 and added a patch
>> in
>> > there, if anybody wants to review it and give me suggestions I'd be
>> happy :)
>>
>> The semantic of tmpbb is not very clear in your patch, it's both the
>> brigade where buckets are saved (for the next call?) and the one that
>> gets passed to the next filter finally.
>> It doesn't look right to me, if tmpbb is to be forwarded in the same
>> pass there is no need to change its buckets' lifetime.
>> Couldn't we ap_save_brigade(f, >holdingbb, >tmpbb, ..) at
>> the end of the filter only?
>>
>
> Thanks for the review, bare in mind that this is my first "big" patch so
> I'd probably need a better grasp of the internals first :) I haven't
> touched holdingbb since I saw that was used elsewhere (in RATE_FULLSPEED),
> but I can try to check it as well. The idea of my patch (that I aimed to)
> is to pass the ctx->tmbbb only if it reaches chunk_size (or EOS is seen)
> and buffer otherwise the buckets in it using ap_save_brigade (waiting for
> the next call to see if chunk_size is reached).
>
> So if I got your point correctly you would use ctx->holdingbb to store the
> buckets (and changing their lifetime possibly) between calls, and tmpbb
> only within the same filter invocation?
>
>
After re-reading the code and I think I got your point Yann, I'll try to
re-work my idea and create a new patch. Thanks!

Luca

Re: [users@httpd] mod_ratelimit working by steps ?

[Luca Toscano]

Hi Yann!

2018-06-04 15:59 GMT+02:00 Yann Ylavic :

> Hi Luca,
>
> On Mon, Jun 4, 2018 at 11:23 AM, Luca Toscano 
> wrote:
> >
> > To keep archives happy: opened
> > https://bz.apache.org/bugzilla/show_bug.cgi?id=62362 and added a patch
> in
> > there, if anybody wants to review it and give me suggestions I'd be
> happy :)
>
> The semantic of tmpbb is not very clear in your patch, it's both the
> brigade where buckets are saved (for the next call?) and the one that
> gets passed to the next filter finally.
> It doesn't look right to me, if tmpbb is to be forwarded in the same
> pass there is no need to change its buckets' lifetime.
> Couldn't we ap_save_brigade(f, >holdingbb, >tmpbb, ..) at
> the end of the filter only?
>

Thanks for the review, bare in mind that this is my first "big" patch so
I'd probably need a better grasp of the internals first :) I haven't
touched holdingbb since I saw that was used elsewhere (in RATE_FULLSPEED),
but I can try to check it as well. The idea of my patch (that I aimed to)
is to pass the ctx->tmbbb only if it reaches chunk_size (or EOS is seen)
and buffer otherwise the buckets in it using ap_save_brigade (waiting for
the next call to see if chunk_size is reached).

So if I got your point correctly you would use ctx->holdingbb to store the
buckets (and changing their lifetime possibly) between calls, and tmpbb
only within the same filter invocation?

Luca

Re: [users@httpd] mod_ratelimit working by steps ?

[Luca Toscano]

2018-05-04 14:46 GMT+02:00 Luca Toscano :

> Hi everybody,
>
> as part of the plan to add more documentation about httpd's internals I am
> trying to debug more tricky bugs (at least for me) reported by our users,
> in order for example to better understand how the filters chain works in
> depth.
>
> This one caught my attention, and after a bit of testing I have some
> follow up questions, if you have time let me know your thoughts :)
>
> 2018-04-19 13:47 GMT+02:00 :
>
>> Hello all,
>>
>> I'm using Apache 2.4.24 on Debian 9 Stable, behind a DSL connection, with
>> an estimated upload capacity of ~130kB/s.
>> I'm trying to limit the bandwidth available to my users (per-connection
>> limit is fine).
>> However, it seems to me that the rate-limit parameter is coarsely grained
>> :
>>
>> - if I set it to 8, users are limited to 8 kB/s
>> - if I set it to 20, or 30, users are limited to 40 kB/s
>> - if I set it to 50, 60 or 80, users are limited to my BW, so ~120 kB/s
>>
>>
> After following up with the user it seems that the issue happens with
> proxied content. So I've set up the following experiment:
>
> - Directory with a 4MB file inside
> - Simple Location that proxies content via mod_proxy_http to a Python
> process running a webserver, capable of returning the same 4MB file
> outlined above.
>
> I tested the rate limit using curl's summary (average Dload speed for
> example).
>
> This is what I gathered:
>
> - when httpd serves the file directly, mod_ratelimit's output filter is
> called once and the bucket brigade contains all the data contained in the
> file. This is probably due to how bucket brigates work when morphing a file
> content?
>
> - when httpd serves the file via mod_proxy, the output filter is called
> multiple times, and each time the buckets are maximum the size of
> ProxyIOBufferSize (8192 by default). Still not completely sure about this
> one, so please let me know if I am totally wrong :)
>
> The main problem is, IIUC, in the output's filter logic that does this: it
> calculates the size of a chunk, based on the rate-limit set in the httpd's
> conf, and then it splits the bucket brigade, if necessary, in buckets of
> that chunk size, interleaving them with FLUSH buckets (and sleeping 200ms).
>
> So a trace of execution with say a chunk size of 8192 would be something
> like:
>
> First call of the filter: 8192 --> FLUSH --> sleep(200ms) --> 8192 --> ...
> -> last chunk (either 8192 or something less).
>
> This happens correctly when httpd serves directly the content, but not
> when proxied:
>
> First call of the filter: 8192 -> FLUSH (no sleep, since do_sleep turns to
> 1 only after the first flush)
>
> Second call of the filter: 8192 -> FLUSH (no sleep)
>
> ...
>
> So one way to alleviate this issue is to move do_sleep to the ctx data
> structure, so if the filter gets called multiple times it will "remember"
> to sleep between flushes (with the assumption that it is allocated for each
> request). It remains the problem that when the rate-limit speed sets a
> chunk size greater than the ProxyIOBufferSize (8192 by default) then the
> client will be rate limited to the speed dictated by the buffer size (for
> example, 8192 should correspond to ~40KB/s).
>
> Without the patch of do_sleep in ctx though, as reported by the user,
> after some rate-limit there won't be any sleep anymore and hence almost no
> ratelimit (only FLUSH buckets that might slow down the overall throughput).
>
> Thanks for reading so far, I hope that what I wrote makes sense. If so,
> I'd document this use case in the mod_ratelimit documentation page and
> possibly submit a patch, otherwise I'll try to study more following up from
> your comments :)
>
>
>
To keep archives happy: opened
https://bz.apache.org/bugzilla/show_bug.cgi?id=62362 and added a patch in
there, if anybody wants to review it and give me suggestions I'd be happy :)

Thanks!

Luca

Layout and best practices for the testing framework

[Luca Toscano]

Hi everybody,

I came up with http://home.apache.org/~elukey/httpd-framework-pr61860.patch
to add some tests for PR 61860. The patch that I have in mind is for
http_protocol, but for the moment the "visible" use case is that an out of
range bytes request that leads to a 416 may include duplicate headers set
via Header always set.

I can see several directories in the repo for the .t tests, like:

- apache, that contains various tests, some of them named "pr".
- modules, that contains specific tests for some modules.
- filter (that could be a subdir of the above, but not sure).
- and finally some generic dirs like "ssl", "php", etc..

Same thing for the supporting httpd.conf's snippets as well.

Do we have some guidelines/conventions about where/how to add tests? I
haven't found anything in our docs up to know, but I might have missed
something.

Thanks!

Luca

Re: [users@httpd] mod_ratelimit working by steps ?

[Luca Toscano]

Hi everybody,

as part of the plan to add more documentation about httpd's internals I am
trying to debug more tricky bugs (at least for me) reported by our users,
in order for example to better understand how the filters chain works in
depth.

This one caught my attention, and after a bit of testing I have some follow
up questions, if you have time let me know your thoughts :)

2018-04-19 13:47 GMT+02:00 :

> Hello all,
>
> I'm using Apache 2.4.24 on Debian 9 Stable, behind a DSL connection, with
> an estimated upload capacity of ~130kB/s.
> I'm trying to limit the bandwidth available to my users (per-connection
> limit is fine).
> However, it seems to me that the rate-limit parameter is coarsely grained :
>
> - if I set it to 8, users are limited to 8 kB/s
> - if I set it to 20, or 30, users are limited to 40 kB/s
> - if I set it to 50, 60 or 80, users are limited to my BW, so ~120 kB/s
>
>
After following up with the user it seems that the issue happens with
proxied content. So I've set up the following experiment:

- Directory with a 4MB file inside
- Simple Location that proxies content via mod_proxy_http to a Python
process running a webserver, capable of returning the same 4MB file
outlined above.

I tested the rate limit using curl's summary (average Dload speed for
example).

This is what I gathered:

- when httpd serves the file directly, mod_ratelimit's output filter is
called once and the bucket brigade contains all the data contained in the
file. This is probably due to how bucket brigates work when morphing a file
content?

- when httpd serves the file via mod_proxy, the output filter is called
multiple times, and each time the buckets are maximum the size of
ProxyIOBufferSize (8192 by default). Still not completely sure about this
one, so please let me know if I am totally wrong :)

The main problem is, IIUC, in the output's filter logic that does this: it
calculates the size of a chunk, based on the rate-limit set in the httpd's
conf, and then it splits the bucket brigade, if necessary, in buckets of
that chunk size, interleaving them with FLUSH buckets (and sleeping 200ms).

So a trace of execution with say a chunk size of 8192 would be something
like:

First call of the filter: 8192 --> FLUSH --> sleep(200ms) --> 8192 --> ...
-> last chunk (either 8192 or something less).

This happens correctly when httpd serves directly the content, but not when
proxied:

First call of the filter: 8192 -> FLUSH (no sleep, since do_sleep turns to
1 only after the first flush)

Second call of the filter: 8192 -> FLUSH (no sleep)

...

So one way to alleviate this issue is to move do_sleep to the ctx data
structure, so if the filter gets called multiple times it will "remember"
to sleep between flushes (with the assumption that it is allocated for each
request). It remains the problem that when the rate-limit speed sets a
chunk size greater than the ProxyIOBufferSize (8192 by default) then the
client will be rate limited to the speed dictated by the buffer size (for
example, 8192 should correspond to ~40KB/s).

Without the patch of do_sleep in ctx though, as reported by the user, after
some rate-limit there won't be any sleep anymore and hence almost no
ratelimit (only FLUSH buckets that might slow down the overall throughput).

Thanks for reading so far, I hope that what I wrote makes sense. If so, I'd
document this use case in the mod_ratelimit documentation page and possibly
submit a patch, otherwise I'll try to study more following up from your
comments :)

Luca

Re: svn commit: r1829430 - /httpd/httpd/patches/2.4.x/core-check_errorlog_dir_syslog.patch

[Luca Toscano]

2018-04-20 16:27 GMT+02:00 Jim Riggs :

> > On 20 Apr 2018, at 08:53, Jim Jagielski  wrote:
> >
> > Sorry for coming in late, but what is the exact issue we are trying to
> solve again? My understanding was that if someone wanted something like
> >
> >   ErrorLog "syslog-httpd.log"
> >
> > that the current implementation would, incorrectly, send the log data to
> syslogd. Is that right?
>
> Luca is working PR 62102 which has to do with "syslog:...:...", but that
> unveiled an inconsistency between core.c and log.c. Before his proposed
> patch in STATUS, core.c is doing a strcmp() for "syslog" whereas log.c is
> doing strncasecmp(). We have been discussing standardizing both on
> strn?cmp(), but that would potentially lead to "breaking" configs that use
> "SYSLOG" rather than "syslog".
>
>
I don't believe that they two things are separate (core/log.c), since the
former checks for ErrorLog's parameter sanity and the latter sets it, so it
would be weird in my opinion if the two logic were different. This is why I
liked your consistency proposal Jim, and applied to my patch. In our docs
we clearly specify to use "syslog" in lowercase, so as far as I can see it
using "SYSLOG" would not be something that people should use..

Luca

Re: svn commit: r1829430 - /httpd/httpd/patches/2.4.x/core-check_errorlog_dir_syslog.patch

[Luca Toscano]

2018-04-19 17:49 GMT+02:00 Jim Riggs :

> Luca -
>
> Here's the same thing standardizing on strn?cmp(). Not that you couldn't
> have done it yourself, but since I had it up, maybe this will save you 30
> seconds. ;-)
>
>
Thanks a lot! I added your last suggestions to r1829626 and also a CHANGES
entry. I tested it on my local environment and it seems working fine, let
me know if everything looks good or if anything is missing.

Luca

Re: So... when should we do 2.4.34? [WAS: Re: Revisit Versioning? (Was: 2.4.3x regression w/SSL vhost configs)]

[Luca Toscano]

2018-04-19 15:40 GMT+02:00 Eric Covener :

> On Thu, Apr 19, 2018 at 9:31 AM, Jim Jagielski  wrote:
> > I agree a case could be made for considering adding an RC stage
> > to our release process... it would require some additional tooling
> > and some sort of additions to ap_release.h but nothing insurmountable.
> >
> > That might be a nice small-and-easily-reversable step to address
> > some of the issues that we've been having lately.
>
> Including burning version numbers!
>

+1

Re: "Most Popular Web Server?"

[Luca Toscano]

Hi Nick,

2018-04-19 10:33 GMT+02:00 Nick Kew <n...@apache.org>:

>
> > On 18 Apr 2018, at 20:00, Luca Toscano <toscano.l...@gmail.com> wrote:
> >
> > Before joining the httpd project as contributor I struggled to find good
> technical sources about how the httpd internals work,
>
> Likewise.  That’s kind-of what motivated me to start writing about it.
>

And I remember reading your book 2/3 times when I wrote my first module at
work, so thanks a lot :)

But that’s not to say it’s any worse than other software projects I’ve
> encountered over the years.
> There’s always a learning curve, and a struggle to find relevant docs.
> OK, things have improved
> a lot since “just google it” became an option, but information still needs
> unearthing.
>
> Are you suggesting httpd is somehow *worse* than other software you’ve
> hacked in terms of
> developer documentation?  In my experience it’s actually a lot better than
> most, due primarily to
> the high standard of API docs in /include/ and in APR, and of course open
> and searchable source.


I agree that we have a good code documentation, what we miss in my opinion
is a high level (but detailed and with examples) up to date overview of how
things stitch together when a request is processed. I can give you a recent
example from my experience, namely
https://bz.apache.org/bugzilla/show_bug.cgi?id=61860. Eric patiently helped
me to track down how the response flows through the output filters (ending
up in a error response eventually), that helped me a lot to understand more
how things works. And it is of course a super simple example for most of
the experienced httpd's devs, but for people like me that are still very
n00b (and ramping up!) this knowledge is a real treasure that makes the
difference between hours of frustration (ending up nowhere) and the same
time spent in coming up with working patches and contribution for the
project.

http://httpd.apache.org/docs/trunk/developer/API.html is a good example
about something that is a "bit" old but still interesting and helpful. The
same up to date thing for 2.4 could be really valuable for anybody that
joins the project (or thinks to do so).

http://httpd.apache.org/docs/trunk/developer/filters.html is great but it
misses examples in my opinion. I recently discovered for example the
.gbdinit goodies that we ship, they could be a straightforward tool to come
up with real data in the docs for a simple debug of a request/response.

Ideally the quality bar that I would love to have across our dev docs is
this one http://httpd.apache.org/docs/trunk/developer/modguide.html, but in
my opinion there is still a a lot of work to do :)


> > My point is: blogging is fine, but before even starting that I'd focus
> on dumping everybody's knowledge in sections of the docs like
> http://httpd.apache.org/docs/trunk/developer. It is boring and less fun
> than writing C code for sure, but I bet that a ton of people would enjoy
> details about how things work. It will be easier for people to spot "liars"
> in the web that focus their marketing strategy only on how httpd is "old"
> and not performant too..
>
> I’ve called out “liars” once or twice.  Or more usually, purveyors of
> “cargo-cult” whose idea
> of Apache is rooted in how things haven’t been since 1997 or so.  But I’m
> not sure they’re
> really the issue.  nginx has risen primarily because it’s a genuine
> solution, and secondarily
> because it’s had the evangelical community that goes with a challenger
> against an
> incumbent.  Now that it’s risen to be a competitor on more equal terms,
> the evangelism
> still has momentum.  Insofar as we care about market share, we could
> respond in kind,
> preferably avoiding the wilder fringe.
>

I think that nginx has risen because it is a great tool, and I like the
fact that there are more "competitors" that challenge the status quo. What
I don't like is when projects that were born standing on the shoulders of
giants focus their marketing on discrediting others to gain popularity.
Without clear docs that can allow anybody to verify what is true and what
not, the end result is a ton of FUD as we are seeing. I would start with
improving our dev docs and spread the word via social media (tweets,
medium, etc..).

Luca

Re: "Most Popular Web Server?"

[Luca Toscano]

My 2c!

2018-04-18 19:21 GMT+02:00 William A Rowe Jr :

> On Wed, Apr 18, 2018 at 11:46 AM, Jim Jagielski  wrote:
> > IMO, this boils down to 2 things:
> >
> >   1. nginx, particularly, does a LOT of promoting, marketing, PR, etc...
> >  We don't. They get to promote their FUD all the time and remain
> >  pretty much unchallenged.
>
> Launched a thread on one aspect we don't have right, and not limited
> to nginx. Another aspect we probably won't solve-for is the tendency
> for new server/proxy solutions to be linux-only, or linux/windows. So
> long as we can abstract it, we don't have any reason to jettison other
> active platform communities which offer enough features.
>
> > Personally, I'd like to see the the PMC take a more active and
> > direct role in addressing #1, maybe w/ monthly blog posts
> > coordinated w/ Sally. It would also be cool to reboot Apache Week
> > (I know it was an external, 3rd party effort) in in conjunction
> > w/ the blog posts or instead of it.
>
> That's a great idea! But the readers are almost exclusively httpd's
> adopters, rarely those who are adopters of other technology.
> Would it help retention of existing admins? Sure, somewhat.
>

Before joining the httpd project as contributor I struggled to find good
technical sources about how the httpd internals work, especially when it
comes to important bits like mpm-event and how its architecture can be
compared with other products. One of my first tasks was to improve the
mpm-event's documentation page, and it took me a ton of time to understand
a very high level overview of it (plus a lot of people patiently tried to
explain to me how things were working). Without good "authoritative"
references a lot of people can write whatever they want on httpd, because
there are too few people that can scan the web and discuss inaccuracies (
https://xkcd.com/386).

I keep struggling with internals in these days, even if I check httpd's
code daily, so I can't imagine somebody not involved in the project that
tries to make a comparison between httpd and product X, when the latter has
a ton of good explanation about how it works in detail (most of the times
with a lot of really explicative graphics attached).

My point is: blogging is fine, but before even starting that I'd focus on
dumping everybody's knowledge in sections of the docs like
http://httpd.apache.org/docs/trunk/developer. It is boring and less fun
than writing C code for sure, but I bet that a ton of people would enjoy
details about how things work. It will be easier for people to spot "liars"
in the web that focus their marketing strategy only on how httpd is "old"
and not performant too..


> Until we shake this notion of "2.6/3.0 considered harmful", the httpd
> project is effectively concluded/no more than a couple tinkerers'
> patch collection.
>

Don't really want to comment on other subjects but only pointing that the
statement, in my opinion, is not true. I saw a lot of people recently
pushing for 2.6, and I believe that everybody's agree that it needs to be
done. We are still trying to find a balance between eagerness of improve
how 2.4 works and its limits, but I am pretty sure we'll find a way soon.

Luca

Re: svn commit: r1829430 - /httpd/httpd/patches/2.4.x/core-check_errorlog_dir_syslog.patch

[Luca Toscano]

Thanks a lot Jim! I like your code change and the extra checks, but I'd
prefer to use strncmp if possible, also in log.c.
Feel free to amend the patch, or I'll do it tomorrow (I forgot the entry in
CHANGES too, your name should be on it :).

Luca

2018-04-18 18:36 GMT+02:00 Jim Riggs :

> Fair enough. I'm fine standardizing either way. strn?cmp() is probably
> more "correct". As it stands, though, the check in core is not actually
> checking everything that log.c will handle.
>
>
> > On 18 Apr 2018, at 10:15, William A Rowe Jr  wrote:
> >
> > On Wed, Apr 18, 2018 at 7:17 AM, Jim Riggs  wrote:
> >> I didn't think of this before, but there is one edge case this would
> miss: if someone (for whatever reason) wants a relative ErrorLog *file*
> named `syslog*', for example `ErrorLog "syslog-httpd.log"' or `ErrorLog
> "syslog.log"'. It appears that this is already broken in server/log.c,
> though. Also, log.c is using str(n)casecmp. The following would make things
> consistent and handle this weird edge case. Thoughts?
> >>
> >> Index: server/core.c
> >> ===
> >> --- server/core.c   (revision 1829431)
> >> +++ server/core.c   (working copy)
> >> @@ -4867,7 +4867,8 @@
> >> static int check_errorlog_dir(apr_pool_t *p, server_rec *s)
> >> {
> >> if (!s->error_fname || s->error_fname[0] == '|'
> >> -|| strcmp(s->error_fname, "syslog") == 0) {
> >> +|| strcasecmp(s->error_fname, "syslog") == 0
> >> +|| strncasecmp(s->error_fname, "syslog:", 7) == 0) {
> >> return APR_SUCCESS;
> >> }
> >> else {
> >> @@ -5281,7 +5282,9 @@
> >> apr_file_printf(out, "ServerRoot: \"%s\"\n", ap_server_root);
> >> tmp = ap_server_root_relative(p, sconf->ap_document_root);
> >> apr_file_printf(out, "Main DocumentRoot: \"%s\"\n", tmp);
> >> -if (s->error_fname[0] != '|' && strcmp(s->error_fname, "syslog")
> != 0)
> >> +if (s->error_fname[0] != '|'
> >> +&& strcasecmp(s->error_fname, "syslog") != 0
> >> +&& strncasecmp(s->error_fname, "syslog:", 7) != 0)
> >> tmp = ap_server_root_relative(p, s->error_fname);
> >> else
> >> tmp = s->error_fname;
> >> @@ -5456,4 +5459,3 @@
> >> core_cmds,/* command apr_table_t */
> >> register_hooks/* register hooks */
> >> };
> >> -
> >> Index: server/log.c
> >> ===
> >> --- server/log.c(revision 1829431)
> >> +++ server/log.c(working copy)
> >> @@ -396,7 +396,8 @@
> >> }
> >>
> >> #ifdef HAVE_SYSLOG
> >> -else if (!strncasecmp(s->error_fname, "syslog", 6)) {
> >> +else if (strcasecmp(s->error_fname, "syslog") == 0
> >> + || strncasecmp(s->error_fname, "syslog:", 7) == 0) {
> >> if ((fname = strchr(s->error_fname, ':'))) {
> >> /* s->error_fname could be [level]:[tag] (see #60525) */
> >> const char *tag;
> >
> > Shouldn't we normalize the use of strcmp instead of strcasecmp?
> > In any case it must be entirely normalized to one or the other.
> >
> > Linux is a case-sensitive OS in the first place, and if configured
> > with SYSLOG: today it is broken when you hit core, which ignores
> > that code path. Since the behavior has always been syslog: I'm
> > not seeing a benefit to case insensitivity.
>
>

Re: [Bug 61860] Headers duplication when 416 status code occurs

[Luca Toscano]

2018-04-09 22:38 GMT+02:00 Luca Toscano <toscano.l...@gmail.com>:

> Hi everybody,
>
> 2018-04-05 7:59 GMT+02:00 <bugzi...@apache.org>:
>
>> https://bz.apache.org/bugzilla/show_bug.cgi?id=61860
>>
>> --- Comment #4 from Luca Toscano <toscano.l...@gmail.com> ---
>> Ok now I think I know what's happening (and I got what Eric was trying to
>> suggest). One of the things that ap_send_error_response does is running
>> ap_run_insert_error_filter, that allows modules to insert their filters
>> (that
>> will be executed). mod_headers does it, specifically it adds this bit:
>>
>> /*
>>  * Make sure we propagate any "Header always" settings on the error
>>  * path through http_protocol.c.
>>  */
>> static apr_status_t ap_headers_error_filter(ap_filter_t *f,
>> apr_bucket_brigade *in)
>>
>> It makes sure that the Headers set via 'always' are in err_headers_out, to
>> allow them to be added in the response. The issue in my opinion is that in
>> ap_send_error_respose we swap r->headers_out with r->err_headers_out, and
>> clear
>> r->err_headers_out (that will be re-populated afterwards). Should we
>> simply do:
>>
>> Index: modules/http/http_protocol.c
>> ===
>> --- modules/http/http_protocol.c(revision 1828234)
>> +++ modules/http/http_protocol.c(working copy)
>> @@ -1262,7 +1262,6 @@
>>  }
>>
>>  if (!r->assbackwards) {
>> -apr_table_t *tmp = r->headers_out;
>>
>>  /* For all HTTP/1.x responses for which we generate the message,
>>   * we need to avoid inheriting the "normal status" header fields
>> @@ -1269,9 +1268,7 @@
>>   * that may have been set by the request handler before the
>>   * error or redirect, except for Location on external redirects.
>>   */
>> -r->headers_out = r->err_headers_out;
>> -r->err_headers_out = tmp;
>> -apr_table_clear(r->err_headers_out);
>> +apr_table_clear(r->headers_out);
>>
>>  if (ap_is_HTTP_REDIRECT(status) || (status == HTTP_CREATED)) {
>>  if ((location != NULL) && *location) {
>>
>>
> I am testing the above patch as possible solution for
> https://bz.apache.org/bugzilla/show_bug.cgi?id=61860, in which a user
> reports that a range error request gets headers duplicated (more
> specifically, all the ones set via Header always). Is there anything big
> that I am missing? I think that it these situations httpd should just use
> r->err_headers_out..
>

Still working on this, for the moment I haven't found any clear regression
when applying the following:

 Index: modules/http/http_protocol.c
===
--- modules/http/http_protocol.c(revision 1828234)
+++ modules/http/http_protocol.c(working copy)
@@ -1262,7 +1262,6 @@
 }

 if (!r->assbackwards) {
-apr_table_t *tmp = r->headers_out;

 /* For all HTTP/1.x responses for which we generate the message,
  * we need to avoid inheriting the "normal status" header fields
@@ -1269,9 +1268,7 @@
  * that may have been set by the request handler before the
  * error or redirect, except for Location on external redirects.
  */
-r->headers_out = r->err_headers_out;
-r->err_headers_out = tmp;
-apr_table_clear(r->err_headers_out);
+apr_table_clear(r->headers_out);

 if (ap_is_HTTP_REDIRECT(status) || (status == HTTP_CREATED)) {
 if ((location != NULL) && *location) {

The above bit seems to solve the issue reported by the user in PR 61860.
This code has been working for a long time so I am wondering if I am
missing any use cases, but IIUC swapping headers_out with err_headers_out
seems to deviate from what's written in httpd.h:

 * The difference between headers_out and err_headers_out is that the
 * latter are printed even on error, and persist across internal
redirects
 * (so the headers printed for ErrorDocument handlers will have them).

Any suggestion is really appreciated :)

Thanks!

Luca

Re: URL's in error pages

[Luca Toscano]

2018-04-12 23:52 GMT+02:00 Yann Ylavic :

> On Thu, Apr 12, 2018 at 11:18 PM, Eric Covener  wrote:
> > On Thu, Apr 12, 2018 at 8:33 AM, Daniel Ruggeri 
> wrote:
> >> Since the encoded form is not very useful for humans, I'd sooner remove
> the URL from the page. As you said, we have access_log. As hesitant as I am
> to suggest Yet Another Directive, I also agree that this change should be
> configurable and defaulted to 'Off' for 2.4... no preference on trunk.
> >
> > I had intended opt-out for 2.4 :/
> >
> > Others?
>
> +1 for opt-out, I'm fine with any encoding (as well as remove) by default
> too.
>

+1 for opt-out in 2.4 and remove for trunk!

Luca

Re: unsubscribe

[Luca Toscano]

As any other Apache project, you can find the instructions about how to
unsubscribe in http://httpd.apache.org/lists.html#http-dev

Luca

2018-04-12 17:35 GMT+02:00 Ray Jender :

> Please remove me from this mailing list!
>

Re: [Bug 61860] Headers duplication when 416 status code occurs

[Luca Toscano]

Hi everybody,

2018-04-05 7:59 GMT+02:00 <bugzi...@apache.org>:

> https://bz.apache.org/bugzilla/show_bug.cgi?id=61860
>
> --- Comment #4 from Luca Toscano <toscano.l...@gmail.com> ---
> Ok now I think I know what's happening (and I got what Eric was trying to
> suggest). One of the things that ap_send_error_response does is running
> ap_run_insert_error_filter, that allows modules to insert their filters
> (that
> will be executed). mod_headers does it, specifically it adds this bit:
>
> /*
>  * Make sure we propagate any "Header always" settings on the error
>  * path through http_protocol.c.
>  */
> static apr_status_t ap_headers_error_filter(ap_filter_t *f,
> apr_bucket_brigade *in)
>
> It makes sure that the Headers set via 'always' are in err_headers_out, to
> allow them to be added in the response. The issue in my opinion is that in
> ap_send_error_respose we swap r->headers_out with r->err_headers_out, and
> clear
> r->err_headers_out (that will be re-populated afterwards). Should we
> simply do:
>
> Index: modules/http/http_protocol.c
> ===
> --- modules/http/http_protocol.c(revision 1828234)
> +++ modules/http/http_protocol.c(working copy)
> @@ -1262,7 +1262,6 @@
>  }
>
>  if (!r->assbackwards) {
> -apr_table_t *tmp = r->headers_out;
>
>  /* For all HTTP/1.x responses for which we generate the message,
>   * we need to avoid inheriting the "normal status" header fields
> @@ -1269,9 +1268,7 @@
>   * that may have been set by the request handler before the
>   * error or redirect, except for Location on external redirects.
>   */
> -r->headers_out = r->err_headers_out;
> -r->err_headers_out = tmp;
> -apr_table_clear(r->err_headers_out);
> +apr_table_clear(r->headers_out);
>
>  if (ap_is_HTTP_REDIRECT(status) || (status == HTTP_CREATED)) {
>  if ((location != NULL) && *location) {
>
>
I am testing the above patch as possible solution for
https://bz.apache.org/bugzilla/show_bug.cgi?id=61860, in which a user
reports that a range error request gets headers duplicated (more
specifically, all the ones set via Header always). Is there anything big
that I am missing? I think that it these situations httpd should just use
r->err_headers_out..

Thanks in advance,

Luca

Re: svn commit: r1827912 - in /httpd/httpd/trunk: CHANGES modules/ssl/ssl_engine_config.c modules/ssl/ssl_engine_init.c modules/ssl/ssl_policies.h modules/ssl/ssl_private.h modules/ssl/update_policies

[Luca Toscano]

All good now thanks!

Luca

2018-04-03 13:49 GMT+02:00 Stefan Eissing <stefan.eiss...@greenbytes.de>:

> My bad. Please try again with r1828220 or later.
>
> Cheers, Stefan
>
> > Am 01.04.2018 um 18:57 schrieb Luca Toscano <toscano.l...@gmail.com>:
> >
> > Hi Stefan
> >
> > 2018-03-28 13:15 GMT+02:00 <ic...@apache.org>:
> > Author: icing
> > Date: Wed Mar 28 11:15:18 2018
> > New Revision: 1827912
> >
> > URL: http://svn.apache.org/viewvc?rev=1827912=rev
> > Log:
> > On the trunk:
> > mod_ssl: add support for TLSv1.3 (tested with OpenSSL v1.1.1-pre3, other
> libs may
> >  need more sugar).
> >
> >
> > Modified:
> > httpd/httpd/trunk/CHANGES
> > httpd/httpd/trunk/modules/ssl/ssl_engine_config.c
> > httpd/httpd/trunk/modules/ssl/ssl_engine_init.c
> > httpd/httpd/trunk/modules/ssl/ssl_policies.h
> > httpd/httpd/trunk/modules/ssl/ssl_private.h
> > httpd/httpd/trunk/modules/ssl/update_policies.py
> >
> >
> >
> >
> > Modified: httpd/httpd/trunk/modules/ssl/ssl_engine_init.c
> > URL: http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/ssl/
> ssl_engine_init.c?rev=1827912=1827911=1827912=diff
> > 
> ==
> > --- httpd/httpd/trunk/modules/ssl/ssl_engine_init.c (original)
> > +++ httpd/httpd/trunk/modules/ssl/ssl_engine_init.c Wed Mar 28 11:15:18
> 2018
> > @@ -601,6 +601,9 @@ static apr_status_t ssl_init_ctx_protoco
> >
> >  #else /* #if OPENSSL_VERSION_NUMBER < 0x1010L */
> >  /* We first determine the maximum protocol version we should
> provide */
> > -if (protocol & SSL_PROTOCOL_TLSV1_2) {
> > +if (SSL_HAVE_PROTOCOL_TLSV1_3 && (protocol & SSL_PROTOCOL_TLSV1_3))
> {
> > +prot = TLS1_3_VERSION;
> > +} else  if (protocol & SSL_PROTOCOL_TLSV1_2) {
> >  prot = TLS1_2_VERSION;
> >  } else if (protocol & SSL_PROTOCOL_TLSV1_1) {
> >  prot = TLS1_1_VERSION;
> > @@ -692,6 +708,9 @@ static apr_status_t ssl_init_ctx_protoco
> >
> >  /* Next we scan for the minimal protocol version we should provide,
> >   * but we do not allow holes between max and min */
> > +if (prot == TLS1_3_VERSION && protocol & SSL_PROTOCOL_TLSV1_2) {
> > +prot = TLS1_2_VERSION;
> > +}
> >  if (prot == TLS1_2_VERSION && protocol & SSL_PROTOCOL_TLSV1_1) {
> >  prot = TLS1_1_VERSION;
> >  }
> >
> >
> > it may be a misconfig from my side, but I get the following with openssl
> 1.1.0f (not TLS 1.3 afaics):
> >
> > ssl_engine_init.c: In function ‘ssl_init_ctx_protocol’:
> > ssl_engine_init.c:690:16: error: ‘TLS1_3_VERSION’ undeclared (first use
> in this function)
> >  prot = TLS1_3_VERSION;
> > ^~
> >
> > Adding the following bits makes everything work:
> >
> > Index: modules/ssl/ssl_engine_init.c
> > ===
> > --- modules/ssl/ssl_engine_init.c (revision 1828144)
> > +++ modules/ssl/ssl_engine_init.c (working copy)
> > @@ -685,9 +685,12 @@
> >
> >  #else /* #if OPENSSL_VERSION_NUMBER < 0x1010L */
> >  /* We first determine the maximum protocol version we should
> provide */
> > +#if SSL_HAVE_PROTOCOL_TLSV1_3
> >  if (SSL_HAVE_PROTOCOL_TLSV1_3 && (protocol & SSL_PROTOCOL_TLSV1_3))
> {
> >  prot = TLS1_3_VERSION;
> > -} else  if (protocol & SSL_PROTOCOL_TLSV1_2) {
> > +} else
> > +#endif
> > +if (protocol & SSL_PROTOCOL_TLSV1_2) {
> >  prot = TLS1_2_VERSION;
> >  } else if (protocol & SSL_PROTOCOL_TLSV1_1) {
> >  prot = TLS1_1_VERSION;
> > @@ -708,9 +711,11 @@
> >
> >  /* Next we scan for the minimal protocol version we should provide,
> >   * but we do not allow holes between max and min */
> > +#if SSL_HAVE_PROTOCOL_TLSV1_3
> >  if (prot == TLS1_3_VERSION && protocol & SSL_PROTOCOL_TLSV1_2) {
> >  prot = TLS1_2_VERSION;
> >  }
> > +#endif
> >  if (prot == TLS1_2_VERSION && protocol & SSL_PROTOCOL_TLSV1_1) {
> >  prot = TLS1_1_VERSION;
> >  }
> >
> >
> > Luca
>
>

Re: svn commit: r1826279 - /httpd/httpd/branches/2.4.x/STATUS

[Luca Toscano]

2018-03-24 17:29 GMT+01:00 Luca Toscano <toscano.l...@gmail.com>:

>
>
> 2018-03-23 19:01 GMT+01:00 Joe Orton <jor...@redhat.com>:
>
>> On Fri, Mar 16, 2018 at 11:50:17AM +0100, Luca Toscano wrote:
>> > From my point of view, adding a comment nearby a directive (except in
>> some
>> > cases like you explained above) should be totally safe and transparent
>> to
>> > the user. I haven't ever thought about the possibility that having a
>> inline
>> > comment could be dangerous, and in my opinion we should enforce this
>> vision
>> > and explicitly document when it is not possible it and why.
>> >
>> > The above is my naive view though (after working on this project for a
>> very
>> > short time) so I'd really like to know what's your angle about not
>> > encouraging inline comments (pretty sure that there are use cases that I
>> > didn't think of, and that might be good to be documented).
>>
>> I'd be fine with making in-line comments 100% safe (stripped)
>> everywhere.  I'd think I'd also be fine with making inline comments a
>> config error in all cases, or increasing the X% of cases where that's an
>> error already.
>>
>> I'm not happy about increasing (but to still below 100%) the places
>> where comments are silently stripped, leaving the remaining places where
>> comments might be a security issue (as in Require host foo#bar).  I'm
>> worried this will *increase* the risk of security issues as users become
>> accustomed to using in-line comments.
>>
>
> Thanks a lot for the explanation, now I completely got what you mean. I
> was convinced that inline comments were safe for all directives, and I
> think that a lot of our users already think this too (I was pinged by a
> colleague that was puzzled about the ServerAlias behavior). I came up with
> the following code patch that introduces a new function in utils.c, heavily
> inspired by your patches (I felt bad to say copy/pasted :) that could help:
>
> http://people.apache.org/~elukey/httpd-trunk-core_
> server_alias_comment.patch
>
> Still not 100% tested, but it seems to work for ServerAlias (might need
> more development time, comments welcome!). Reviewing all the directives to
> apply this though might become a big burden, and potentially introduce new
> bugs in 2.4 that we don't want. On the other side, the simplest solution of
> raising an error when inline comments are used might be better, but we'd
> risk to break existing working configurations when users upgrade to the new
> release..
>
> I don't have a strong position on this, I am dumping thoughts more than
> giving solutions, really interested in what others think.
>

If anybody still has patience and wants to review some code, I tried to
update my patch to swap Joe's changes for forward_dns_check_authorization
with a call to ap_getword_conf_nocomment:

http://people.apache.org/~elukey/httpd-trunk-core_server_alias_comment.patch

Tried to test it with 'Require forward-dns' and it seems working fine..

Luca