headers.t error on 2.4 HEAD

[Jim Jagielski]

On HEAD of httpd-2.4, the test framework fails on headers.t:

   header: [Foo Bar] vs [Foo Bar]
   # testing : compare header Hello value
   # expected: 'Foo Bar'
   # received: 'Foo Bar'
   not ok 3

I am guessing that this is due to the header work being
in some sort of intermediate state of development?? Or
has the test framework not yet been updated??

Re: how to build httpd with profile-guided-optimization?

[Reindl Harald]

since there are already higly optimized parts with "-O3 -fPIC 
-funswitch-loops -minline-all-stringops -fno-strict-aliasing -flto 
-ffat-lto-objects -fuse-ld=gold -fuse-linker-plugin" even if you biuld 
with -Os (which is a good combination when performance critical code is 
optimized while the rest not bloated) i guess that parts could be 
enhanced by the implicit "-funroll-loops"


as httpd typically has similar operations for every website it could be 
easier to profile than PHP where you really need to profile your 
application


PGO enables the options below and our PHP build with PGO is 0.8 MB 
smaller (the main binary while anything is built as loadable extension) 
than without while gives around 7-10% performance gain and the 
profiling-code running through a complete website in a loop should be 
re-usable for httpd too

__

-fprofile-use=
Enable profile feedback-directed optimizations, and the following 
optimizations which are generally profitable only with profile feedback 
available: -fbranch-probabilities, -fvpt, -funroll-loops, -fpeel-loops, 
-ftracer, -ftree-vectorize, and ftree-loop-distribute-patterns.


Before you can use this option, you must first generate profiling 
information. See Optimize Options, for information about the 
-fprofile-generate option.


Am 11.10.2016 um 13:32 schrieb Reindl Harald:

https://en.wikipedia.org/wiki/Profile-guided_optimization

for PHP it's easy because the makefiles support it directly

make %{?_smp_mflags} prof-gen
/usr/bin/bash /rpmbuild/PHP-PGO/profile.sh --php_build $PWD
make prof-clean
make %{?_smp_mflags} prof-use
_

i tried to replicate that for httpd but obviously CFLAGS after
./configure are completly ignored but you need "-fprofile-generate" for
the first make and "-fprofile-generate" after benchmark the application
for the second build which means that the stuff belows looks not
terrible wrong but don't work at all

export CFLAGS="$COMPILERFLAGS -fprofile-generate"
export CXXFLAGS="$COMPILERFLAGS -fprofile-generate"
CCACHE_DISABLE=1 make %{?_smp_mflags} PROF_FLAGS="-fprofile-generate all"
/usr/bin/bash /rpmbuild/PHP-PGO/profile.sh --httpd_build $PWD
find . -name \*.lo -o -name \*.o | xargs rm -f
find . -name \*.la -o -name \*.a | xargs rm -f
find . -name \*.so | xargs rm -f
rm -f httpd libs/*
export CFLAGS="$COMPILERFLAGS -fprofile-use"
export CXXFLAGS="$COMPILERFLAGS -fprofile-use"
CCACHE_DISABLE=1 make %{?_smp_mflags} PROF_FLAGS="-fprofile-use all"

Re: Loads of dup symbols in h2_util.c and h2_proxy_util.c

[Jim Jagielski]

Bringo!

https://www.youtube.com/watch?v=4MpyHBoiTwk

> On Oct 11, 2016, at 10:35 AM, Stefan Eissing  
> wrote:
> 
> Fixed (hopefully ;-)
> 
> -Stefan
> 
>> Am 11.10.2016 um 15:59 schrieb Jim Jagielski :
>> 
>> Looks like 2 were missed :)
>> 
>> duplicate symbol _h2_proxy_res_ignore_header in:
>>   modules/http2/.libs/libmod_proxy_http2.a(h2_proxy_util.o)
>>   modules/http2/.libs/libmod_http2.a(h2_util.o)
>> duplicate symbol _h2_util_ngheader_make_req in:
>>   modules/http2/.libs/libmod_proxy_http2.a(h2_proxy_util.o)
>>   modules/http2/.libs/libmod_http2.a(h2_util.o)
>> ld: 2 duplicate symbols for architecture x86_64
>> clang: error: linker command failed with exit code 1 (use -v to see 
>> invocation)
>> 
>> Thx!!
>> 
>>> On Oct 11, 2016, at 9:42 AM, Stefan Eissing  
>>> wrote:
>>> 
>>> Changed. If you could verify that current trunk + 2.4.x do not have this 
>>> issue any more, that'd be nice.
>>> 
>>> -Stefan
>>> 
 Am 11.10.2016 um 15:13 schrieb Jim Jagielski :
 
 ... and on trunk too it seems...
> On Oct 11, 2016, at 9:05 AM, Jim Jagielski  wrote:
> 
> Forgot to mention: This is on HEAD httpd-2.4
> 
>> On Oct 11, 2016, at 8:50 AM, Jim Jagielski  wrote:
>> 
>> There are a bunch of dup symbols popping up during link; these
>> symbols exist in both h2_util.c and h2_proxy_util.c.
>> 
>> Note: this is AFTER ensuring a 'make extraclean'...
> 
 
>>> 
>> 
> 

Re: Loads of dup symbols in h2_util.c and h2_proxy_util.c

[Stefan Eissing]

Fixed (hopefully ;-)

-Stefan

> Am 11.10.2016 um 15:59 schrieb Jim Jagielski :
> 
> Looks like 2 were missed :)
> 
> duplicate symbol _h2_proxy_res_ignore_header in:
>modules/http2/.libs/libmod_proxy_http2.a(h2_proxy_util.o)
>modules/http2/.libs/libmod_http2.a(h2_util.o)
> duplicate symbol _h2_util_ngheader_make_req in:
>modules/http2/.libs/libmod_proxy_http2.a(h2_proxy_util.o)
>modules/http2/.libs/libmod_http2.a(h2_util.o)
> ld: 2 duplicate symbols for architecture x86_64
> clang: error: linker command failed with exit code 1 (use -v to see 
> invocation)
> 
> Thx!!
> 
>> On Oct 11, 2016, at 9:42 AM, Stefan Eissing  
>> wrote:
>> 
>> Changed. If you could verify that current trunk + 2.4.x do not have this 
>> issue any more, that'd be nice.
>> 
>> -Stefan
>> 
>>> Am 11.10.2016 um 15:13 schrieb Jim Jagielski :
>>> 
>>> ... and on trunk too it seems...
 On Oct 11, 2016, at 9:05 AM, Jim Jagielski  wrote:
 
 Forgot to mention: This is on HEAD httpd-2.4
 
> On Oct 11, 2016, at 8:50 AM, Jim Jagielski  wrote:
> 
> There are a bunch of dup symbols popping up during link; these
> symbols exist in both h2_util.c and h2_proxy_util.c.
> 
> Note: this is AFTER ensuring a 'make extraclean'...
 
>>> 
>> 
> 

Re: Loads of dup symbols in h2_util.c and h2_proxy_util.c

[Jim Jagielski]

Looks like 2 were missed :)

duplicate symbol _h2_proxy_res_ignore_header in:
modules/http2/.libs/libmod_proxy_http2.a(h2_proxy_util.o)
modules/http2/.libs/libmod_http2.a(h2_util.o)
duplicate symbol _h2_util_ngheader_make_req in:
modules/http2/.libs/libmod_proxy_http2.a(h2_proxy_util.o)
modules/http2/.libs/libmod_http2.a(h2_util.o)
ld: 2 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Thx!!

> On Oct 11, 2016, at 9:42 AM, Stefan Eissing  
> wrote:
> 
> Changed. If you could verify that current trunk + 2.4.x do not have this 
> issue any more, that'd be nice.
> 
> -Stefan
> 
>> Am 11.10.2016 um 15:13 schrieb Jim Jagielski :
>> 
>> ... and on trunk too it seems...
>>> On Oct 11, 2016, at 9:05 AM, Jim Jagielski  wrote:
>>> 
>>> Forgot to mention: This is on HEAD httpd-2.4
>>> 
 On Oct 11, 2016, at 8:50 AM, Jim Jagielski  wrote:
 
 There are a bunch of dup symbols popping up during link; these
 symbols exist in both h2_util.c and h2_proxy_util.c.
 
 Note: this is AFTER ensuring a 'make extraclean'...
>>> 
>> 
> 

Re: Loads of dup symbols in h2_util.c and h2_proxy_util.c

[Stefan Eissing]

Changed. If you could verify that current trunk + 2.4.x do not have this issue 
any more, that'd be nice.

-Stefan

> Am 11.10.2016 um 15:13 schrieb Jim Jagielski :
> 
> ... and on trunk too it seems...
>> On Oct 11, 2016, at 9:05 AM, Jim Jagielski  wrote:
>> 
>> Forgot to mention: This is on HEAD httpd-2.4
>> 
>>> On Oct 11, 2016, at 8:50 AM, Jim Jagielski  wrote:
>>> 
>>> There are a bunch of dup symbols popping up during link; these
>>> symbols exist in both h2_util.c and h2_proxy_util.c.
>>> 
>>> Note: this is AFTER ensuring a 'make extraclean'...
>> 
> 

Re: Loads of dup symbols in h2_util.c and h2_proxy_util.c

[Jim Jagielski]

... and on trunk too it seems...
> On Oct 11, 2016, at 9:05 AM, Jim Jagielski  wrote:
> 
> Forgot to mention: This is on HEAD httpd-2.4
> 
>> On Oct 11, 2016, at 8:50 AM, Jim Jagielski  wrote:
>> 
>> There are a bunch of dup symbols popping up during link; these
>> symbols exist in both h2_util.c and h2_proxy_util.c.
>> 
>> Note: this is AFTER ensuring a 'make extraclean'...
> 

Re: Loads of dup symbols in h2_util.c and h2_proxy_util.c

[Stefan Eissing]

Fix incoming.

> Am 11.10.2016 um 15:05 schrieb Jim Jagielski :
> 
> Forgot to mention: This is on HEAD httpd-2.4
> 
>> On Oct 11, 2016, at 8:50 AM, Jim Jagielski  wrote:
>> 
>> There are a bunch of dup symbols popping up during link; these
>> symbols exist in both h2_util.c and h2_proxy_util.c.
>> 
>> Note: this is AFTER ensuring a 'make extraclean'...
> 

Re: Loads of dup symbols in h2_util.c and h2_proxy_util.c

[Jim Jagielski]

Forgot to mention: This is on HEAD httpd-2.4

> On Oct 11, 2016, at 8:50 AM, Jim Jagielski  wrote:
> 
> There are a bunch of dup symbols popping up during link; these
> symbols exist in both h2_util.c and h2_proxy_util.c.
> 
> Note: this is AFTER ensuring a 'make extraclean'...

Loads of dup symbols in h2_util.c and h2_proxy_util.c

[Jim Jagielski]

There are a bunch of dup symbols popping up during link; these
symbols exist in both h2_util.c and h2_proxy_util.c.

Note: this is AFTER ensuring a 'make extraclean'...

how tu build httpd with profile-guided-optimization?

[Reindl Harald]

https://en.wikipedia.org/wiki/Profile-guided_optimization

for PHP it's easy because the makefiles support it directly

make %{?_smp_mflags} prof-gen
/usr/bin/bash /rpmbuild/PHP-PGO/profile.sh --php_build $PWD
make prof-clean
make %{?_smp_mflags} prof-use
_

i tried to replicate that for httpd but obviously CFLAGS after 
./configure are completly ignored but you need "-fprofile-generate" for 
the first make and "-fprofile-generate" after benchmark the application 
for the second build which means that the stuff belows looks not 
terrible wrong but don't work at all


export CFLAGS="$COMPILERFLAGS -fprofile-generate"
export CXXFLAGS="$COMPILERFLAGS -fprofile-generate"
CCACHE_DISABLE=1 make %{?_smp_mflags} PROF_FLAGS="-fprofile-generate all"
/usr/bin/bash /rpmbuild/PHP-PGO/profile.sh --httpd_build $PWD
find . -name \*.lo -o -name \*.o | xargs rm -f
find . -name \*.la -o -name \*.a | xargs rm -f
find . -name \*.so | xargs rm -f
rm -f httpd libs/*
export CFLAGS="$COMPILERFLAGS -fprofile-use"
export CXXFLAGS="$COMPILERFLAGS -fprofile-use"
CCACHE_DISABLE=1 make %{?_smp_mflags} PROF_FLAGS="-fprofile-use all"