Re: Travis Matrix (was: Re: [PATCH] wurfl device detection build fixes and dummy library)

[Willy Tarreau]

On Fri, Jun 14, 2019 at 01:34:34AM +0500,  ??? wrote:
> well, I am going to play with cron jobs say "after haproxy-2.0 release"

OK, that works for me!

> as for being good citizens, when I look at
> 
> https://travis-ci.org/openssl/openssl/builds
> 
> (25k builds 1hr each ), I'm sure we are good citizens. really :)

Indeed, it could be said we have some margin :-)

Willy

Re: Travis Matrix (was: Re: [PATCH] wurfl device detection build fixes and dummy library)

[Илья Шипицин]

пт, 14 июн. 2019 г. в 01:27, Willy Tarreau :

> On Thu, Jun 13, 2019 at 10:28:19PM +0500,  ??? wrote:
> > > > or we can drop boringssl build in favour of Manu :)
> > >
> > > I suggest to drop BoringSSL. It's not intended to be used outside of
> > > Google anyway. So if it breaks the user gets to keep both pieces and
> can
> > > report the issue in the tracker or on the list.
> > >
> >
> > recently, I've found that if you wish to specify custom cipher suites in
> > CloudFlare, you need to use BoringSSL syntax.
> > so there's some use of it. same for LibreSSL
>
> I'd say that I'm in favor of testing what doesn't add effort nor cost.
> At the moment BoringSSL represents a cost for the infrastructure and the
> build time, but if we can use a cron job with it it should be a good
> option. In this case we could possibly do the same with the least
> common combinations (e.g. linux+no option, older libressl builds) and
> be good citizens with Travis and save on build time.
>
> Just a suggestion anyway, I don't have strong feelings on this.
>

well, I am going to play with cron jobs say "after haproxy-2.0 release"

as for being good citizens, when I look at

https://travis-ci.org/openssl/openssl/builds

(25k builds 1hr each ), I'm sure we are good citizens. really :)



>
> Willy
>

Re: Travis Matrix (was: Re: [PATCH] wurfl device detection build fixes and dummy library)

[Willy Tarreau]

On Thu, Jun 13, 2019 at 10:28:19PM +0500,  ??? wrote:
> > > or we can drop boringssl build in favour of Manu :)
> >
> > I suggest to drop BoringSSL. It's not intended to be used outside of
> > Google anyway. So if it breaks the user gets to keep both pieces and can
> > report the issue in the tracker or on the list.
> >
> 
> recently, I've found that if you wish to specify custom cipher suites in
> CloudFlare, you need to use BoringSSL syntax.
> so there's some use of it. same for LibreSSL

I'd say that I'm in favor of testing what doesn't add effort nor cost.
At the moment BoringSSL represents a cost for the infrastructure and the
build time, but if we can use a cron job with it it should be a good
option. In this case we could possibly do the same with the least
common combinations (e.g. linux+no option, older libressl builds) and
be good citizens with Travis and save on build time.

Just a suggestion anyway, I don't have strong feelings on this.

Willy

Re: Travis Matrix (was: Re: [PATCH] wurfl device detection build fixes and dummy library)

[Илья Шипицин]

чт, 13 июн. 2019 г. в 22:25, Tim Düsterhus :

> Ilya,
> Willy,
>
> Am 13.06.19 um 19:11 schrieb Илья Шипицин:
> > чт, 13 июн. 2019 г. в 22:07, Willy Tarreau :
> >
> >> On Thu, Jun 13, 2019 at 10:01:52PM +0500,  ??? wrote:
> >>> "everything enabled" is impossible. 51degrees may be enabled in two
> >>> mutually exclusive ways. it doubles number of build configurations.
> >>
> >> It's not big deal, the haproxy-specific code remains the same and only
> >> the 51D-specific one differs, so once Ben tells you "this one or that
> >> one is more relevant", we stick to it and that's fine.
>
> I agree with Willy here. The dummy library primarily exists to make sure
> the appropriate functiions exist to link against. Just select one (I
> guess the 'pattern' based one) and we should be good. The dummy library
> is an approximation anyway.
>
> >> Thinking about the boringssl mess which takes ages to rebuild, I don't
> >> know if it's possible to script some form of "once every X times" or
> >> alternatively "build only when random()%10==0". That could also help
> >> preserving travis resources while still reporting once in a while if
> >> we broke anything there. I'd also say that boringssl is constantly
> >> built by Manu, I wouldn't be surprised if he detected our breakage
> >> faster than travis :-)
> >>
> >
> > there are couple of options:
> >
> > 1) we can use ccache (and cache it between builds). however, ccache is
> > buggy itself, I turned it "off" already for several projects
> >
> > 2) we can setup special "cron" builds, for example it will build haproxy
> +
> > boringssl once a day.
> >
> > or we can drop boringssl build in favour of Manu :)
>
> I suggest to drop BoringSSL. It's not intended to be used outside of
> Google anyway. So if it breaks the user gets to keep both pieces and can
> report the issue in the tracker or on the list.
>

recently, I've found that if you wish to specify custom cipher suites in
CloudFlare, you need to use BoringSSL syntax.
so there's some use of it. same for LibreSSL



>
> Best regards
> Tim Düsterhus
>

Re: Travis Matrix (was: Re: [PATCH] wurfl device detection build fixes and dummy library)

[Tim Düsterhus]

Ilya,
Willy,

Am 13.06.19 um 19:11 schrieb Илья Шипицин:
> чт, 13 июн. 2019 г. в 22:07, Willy Tarreau :
> 
>> On Thu, Jun 13, 2019 at 10:01:52PM +0500,  ??? wrote:
>>> "everything enabled" is impossible. 51degrees may be enabled in two
>>> mutually exclusive ways. it doubles number of build configurations.
>>
>> It's not big deal, the haproxy-specific code remains the same and only
>> the 51D-specific one differs, so once Ben tells you "this one or that
>> one is more relevant", we stick to it and that's fine.

I agree with Willy here. The dummy library primarily exists to make sure
the appropriate functiions exist to link against. Just select one (I
guess the 'pattern' based one) and we should be good. The dummy library
is an approximation anyway.

>> Thinking about the boringssl mess which takes ages to rebuild, I don't
>> know if it's possible to script some form of "once every X times" or
>> alternatively "build only when random()%10==0". That could also help
>> preserving travis resources while still reporting once in a while if
>> we broke anything there. I'd also say that boringssl is constantly
>> built by Manu, I wouldn't be surprised if he detected our breakage
>> faster than travis :-)
>>
> 
> there are couple of options:
> 
> 1) we can use ccache (and cache it between builds). however, ccache is
> buggy itself, I turned it "off" already for several projects
> 
> 2) we can setup special "cron" builds, for example it will build haproxy +
> boringssl once a day.
> 
> or we can drop boringssl build in favour of Manu :)

I suggest to drop BoringSSL. It's not intended to be used outside of
Google anyway. So if it breaks the user gets to keep both pieces and can
report the issue in the tracker or on the list.

Best regards
Tim Düsterhus

Re: Travis Matrix (was: Re: [PATCH] wurfl device detection build fixes and dummy library)

[Willy Tarreau]

On Thu, Jun 13, 2019 at 10:01:52PM +0500,  ??? wrote:
> "everything enabled" is impossible. 51degrees may be enabled in two
> mutually exclusive ways. it doubles number of build configurations.

It's not big deal, the haproxy-specific code remains the same and only
the 51D-specific one differs, so once Ben tells you "this one or that
one is more relevant", we stick to it and that's fine.

Thinking about the boringssl mess which takes ages to rebuild, I don't
know if it's possible to script some form of "once every X times" or
alternatively "build only when random()%10==0". That could also help
preserving travis resources while still reporting once in a while if
we broke anything there. I'd also say that boringssl is constantly
built by Manu, I wouldn't be surprised if he detected our breakage
faster than travis :-)

Willy

Re: Travis Matrix (was: Re: [PATCH] wurfl device detection build fixes and dummy library)

[Willy Tarreau]

On Thu, Jun 13, 2019 at 06:28:09PM +0200, Tim Düsterhus wrote:
> I'm unhappy with that patch:
> 
> a) It makes unrelated changes to the OpenSSL version (that should be a
> separate patch).

I'm also at fault here because I saw this and didn't bother that much,
being busy on other stuff :-/

> b) It removed the "gcc without any flags" build and instead replaced it
> with "gcc with all flags, but a separate 51d from all others".

I didn't notice this one.

> I consider the "X without any flags" very useful to detect missing
> preprocessor guards and code accidentally relying on a build flag.

I agree, and in "without any flags" I'd even disable threads as we
broke the thread-less builds quite a few times in the past.

> I suggest to draft a real plan on how the tests should proceed, taking
> into account to not abuse Travis' free service, before making any more
> changes to the configuration.
> 
> I suggest:
> 
> - Linux + gcc   with everything disabled.
> - Linux + clang with everything disabled.
> - Linux + gcc   with everything enabled (incl. device detection and
> prometheus).
> - Linux + clang with everything enabled (incl. device detection and
> prometheus).
> - Linux + gcc   with ONLY SSL enabled for each of the various SSL libraries.
> - Mac + clang   with everything from core HAProxy enabled (no device
> detection and prometheus).
> - Windows   with everything from core HAProxy enabled (no device
> detection and prometheus).
> 
> That way the edge cases (everything enabled and everything disabled) are
> systematically tested. The less important platforms (Windows / Mac;
> everything that is not the default OpenSSL) get some basic exposure
> without skyrocketing build times, because things are redundantly tested
> (e.g. no need to test compression with each SSL library, that's why only
> SSL should be enabled for those).

I tend to think that this proposal is quite reasonable.

Willy

Re: Travis Matrix (was: Re: [PATCH] wurfl device detection build fixes and dummy library)

[Илья Шипицин]

чт, 13 июн. 2019 г. в 21:28, Tim Düsterhus :

> Ilya,
>
> (removed Ben and Christopher from Cc, as this no longer about 51d)
>
> Am 13.06.19 um 17:01 schrieb Илья Шипицин:
> > please find "travis-ci + 51degree" patch attached.
>
> I'm unhappy with that patch:
>
> a) It makes unrelated changes to the OpenSSL version (that should be a
> separate patch).
> b) It removed the "gcc without any flags" build and instead replaced it
> with "gcc with all flags, but a separate 51d from all others".
>
> I consider the "X without any flags" very useful to detect missing
> preprocessor guards and code accidentally relying on a build flag.
>
> I'm aware that there still is "clang without any flags", but there
> really should be "clang without any flags" and "gcc without any flags"
> as history has shown that they sometimes do things differently.
>
> All in all the Travis configuration is pretty unorganized by now. It
> "randomly" mashes together various options so that everything is somehow
> tested at least once, but there is no real logic behind it. As an
> example: Why is the Prometheus test added to LibreSSL 2.7.5? Why not
> BoringSSL?
>
> I suggest to draft a real plan on how the tests should proceed, taking
> into account to not abuse Travis' free service, before making any more
> changes to the configuration.
>
> I suggest:
>
> - Linux + gcc   with everything disabled.
> - Linux + clang with everything disabled.
> - Linux + gcc   with everything enabled (incl. device detection and
> prometheus).
>


"everything enabled" is impossible. 51degrees may be enabled in two
mutually exclusive ways. it doubles number of build configurations.



> - Linux + clang with everything enabled (incl. device detection and
> prometheus).
> - Linux + gcc   with ONLY SSL enabled for each of the various SSL
> libraries.
> - Mac + clang   with everything from core HAProxy enabled (no device
> detection and prometheus).
> - Windows   with everything from core HAProxy enabled (no device
> detection and prometheus).
>
> That way the edge cases (everything enabled and everything disabled) are
> systematically tested. The less important platforms (Windows / Mac;
> everything that is not the default OpenSSL) get some basic exposure
> without skyrocketing build times, because things are redundantly tested
> (e.g. no need to test compression with each SSL library, that's why only
> SSL should be enabled for those).
>
> Best regards
> Tim Düsterhus
>

Re: Travis Matrix (was: Re: [PATCH] wurfl device detection build fixes and dummy library)

[Илья Шипицин]

чт, 13 июн. 2019 г. в 21:28, Tim Düsterhus :

> Ilya,
>
> (removed Ben and Christopher from Cc, as this no longer about 51d)
>
> Am 13.06.19 um 17:01 schrieb Илья Шипицин:
> > please find "travis-ci + 51degree" patch attached.
>
> I'm unhappy with that patch:
>
> a) It makes unrelated changes to the OpenSSL version (that should be a
> separate patch).
> b) It removed the "gcc without any flags" build and instead replaced it
> with "gcc with all flags, but a separate 51d from all others".
>
> I consider the "X without any flags" very useful to detect missing
> preprocessor guards and code accidentally relying on a build flag.
>
> I'm aware that there still is "clang without any flags", but there
> really should be "clang without any flags" and "gcc without any flags"
> as history has shown that they sometimes do things differently.
>
> All in all the Travis configuration is pretty unorganized by now. It
> "randomly" mashes together various options so that everything is somehow
> tested at least once, but there is no real logic behind it. As an
> example: Why is the Prometheus test added to LibreSSL 2.7.5? Why not
> BoringSSL?
>

I'd like to add PCRE2 and SLZ to future builds
also, I'd like to add more clang sanitizers (currently we use only ASAN and
it is extremely useful)

also, I'm not sure how many LibreSSL versions do we actually need (I taken
three of them)

as for gcc, I'm not happy with it's sanitizers, but maybe I'll add some
later. that's why we have much
more clang builds.

also, I'm not very happy with current BoringSSL caching (it caches only
downloads, and it seems
that S3 cache is almost the same as clean checkout)



>
> I suggest to draft a real plan on how the tests should proceed, taking
> into account to not abuse Travis' free service, before making any more
> changes to the configuration.
>


sure. I do not mind


>
> I suggest:
>
> - Linux + gcc   with everything disabled.
> - Linux + clang with everything disabled.
> - Linux + gcc   with everything enabled (incl. device detection and
> prometheus).
> - Linux + clang with everything enabled (incl. device detection and
> prometheus).
> - Linux + gcc   with ONLY SSL enabled for each of the various SSL
> libraries.
> - Mac + clang   with everything from core HAProxy enabled (no device
> detection and prometheus).
> - Windows   with everything from core HAProxy enabled (no device
> detection and prometheus).
>
> That way the edge cases (everything enabled and everything disabled) are
> systematically tested. The less important platforms (Windows / Mac;
> everything that is not the default OpenSSL) get some basic exposure
> without skyrocketing build times, because things are redundantly tested
> (e.g. no need to test compression with each SSL library, that's why only
> SSL should be enabled for those).
>
> Best regards
> Tim Düsterhus
>

Travis Matrix (was: Re: [PATCH] wurfl device detection build fixes and dummy library)

[Tim Düsterhus]

Ilya,

(removed Ben and Christopher from Cc, as this no longer about 51d)

Am 13.06.19 um 17:01 schrieb Илья Шипицин:
> please find "travis-ci + 51degree" patch attached.

I'm unhappy with that patch:

a) It makes unrelated changes to the OpenSSL version (that should be a
separate patch).
b) It removed the "gcc without any flags" build and instead replaced it
with "gcc with all flags, but a separate 51d from all others".

I consider the "X without any flags" very useful to detect missing
preprocessor guards and code accidentally relying on a build flag.

I'm aware that there still is "clang without any flags", but there
really should be "clang without any flags" and "gcc without any flags"
as history has shown that they sometimes do things differently.

All in all the Travis configuration is pretty unorganized by now. It
"randomly" mashes together various options so that everything is somehow
tested at least once, but there is no real logic behind it. As an
example: Why is the Prometheus test added to LibreSSL 2.7.5? Why not
BoringSSL?

I suggest to draft a real plan on how the tests should proceed, taking
into account to not abuse Travis' free service, before making any more
changes to the configuration.

I suggest:

- Linux + gcc   with everything disabled.
- Linux + clang with everything disabled.
- Linux + gcc   with everything enabled (incl. device detection and
prometheus).
- Linux + clang with everything enabled (incl. device detection and
prometheus).
- Linux + gcc   with ONLY SSL enabled for each of the various SSL libraries.
- Mac + clang   with everything from core HAProxy enabled (no device
detection and prometheus).
- Windows   with everything from core HAProxy enabled (no device
detection and prometheus).

That way the edge cases (everything enabled and everything disabled) are
systematically tested. The less important platforms (Windows / Mac;
everything that is not the default OpenSSL) get some basic exposure
without skyrocketing build times, because things are redundantly tested
(e.g. no need to test compression with each SSL library, that's why only
SSL should be enabled for those).

Best regards
Tim Düsterhus

RE: [PATCH] wurfl device detection build fixes and dummy library

[Ben Shillito]

Hi Willy,

Great, thanks.

Yeah that makes total sense. Don't want warnings that can't be solved.

Regards,

Ben Shillito
Developer
O: +44 1183 287152
E: b...@51degrees.com
T: @51Degrees

-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu]
Sent: 13 June 2019 17:06
To: Ben Shillito 
Cc:  ??? ; Christopher Faulet 
; HAProxy 
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library

On Thu, Jun 13, 2019 at 03:57:04PM +, Ben Shillito wrote:
> Also, after thinking earlier about making it more obvious that it is a
> dummy library, I have attached a patch which adds "(dummy library)" to
> the output of REGISTER_BUILD_OPTS macro.

Ah perfect, I also wanted to do it but considered it of lower importance than 
the remaining stuff so it could be postponed. I've merged it now, thank you.

> I thought about pushing a warning when the dummy library is used just
> to be super obvious. Don't know what you think of that?

No it's not a good idea. We're trying to make sure that *any* single warning 
must be addressable by the end user, otherwise people get used to seeing them 
and to ignore them.

If people build with the dummy lib and complain, they'll send their haproxy 
-vv, you see "dummy" there it becomes "you don't need to run with this, please 
prove the problem still happens without", end of the story.

I asked about these libs for convenience for *developers* not for users.
But if it causes warnings in all our builds, it will end up being disabled 
which is counter productive.

Cheers,
Willy
This email and any attachments are confidential and may also be privileged. If 
you are not the named recipient, please notify the sender immediately and do 
not disclose, use, store or copy the information contained herein. This is an 
email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 
118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees.

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

On Thu, Jun 13, 2019 at 03:57:04PM +, Ben Shillito wrote:
> Also, after thinking earlier about making it more obvious that it is a dummy
> library, I have attached a patch which adds "(dummy library)" to the output
> of REGISTER_BUILD_OPTS macro.

Ah perfect, I also wanted to do it but considered it of lower importance
than the remaining stuff so it could be postponed. I've merged it now, thank
you.

> I thought about pushing a warning when the dummy library is used just to be
> super obvious. Don't know what you think of that?

No it's not a good idea. We're trying to make sure that *any* single
warning must be addressable by the end user, otherwise people get used
to seeing them and to ignore them.

If people build with the dummy lib and complain, they'll send their
haproxy -vv, you see "dummy" there it becomes "you don't need to run
with this, please prove the problem still happens without", end of the
story.

I asked about these libs for convenience for *developers* not for users.
But if it causes warnings in all our builds, it will end up being disabled
which is counter productive.

Cheers,
Willy

RE: [PATCH] wurfl device detection build fixes and dummy library

[Ben Shillito]

Same here. I can't see anything related to 51Degrees in there.

I will leave this for now, but let me know if either of you notice anything I 
need to look into.

Also, after thinking earlier about making it more obvious that it is a dummy 
library, I have attached a patch which adds "(dummy library)" to the output of 
REGISTER_BUILD_OPTS macro.

I thought about pushing a warning when the dummy library is used just to be 
super obvious. Don't know what you think of that?

Thanks,

Ben Shillito
Developer
O: +44 1183 287152
E: b...@51degrees.com
T: @51Degrees

-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu]
Sent: 13 June 2019 16:30
To: Ben Shillito 
Cc:  ??? ; Christopher Faulet 
; HAProxy 
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library

On Thu, Jun 13, 2019 at 03:21:36PM +, Ben Shillito wrote:
> Thanks both,
>
> Ilya, I will take a look at that now.

I suspect it's totally unrelated, the failure was on this reg test and may be 
just a race :

## Test case: reg-tests/checks/4be_1srv_health_checks.vtc ## ## test 
results in: "/tmp/haregtests-2019-06-13_15-05-08.R3rhNr/vtc.8210.558cb932"
 h11.5 CLI expect failed ~ "# be_id be_name srv_id srv_name srv_addr 
srv_op_state srv_admin_state srv_uweight srv_iweight srv_time_since_last_change 
srv_check_status srv_check_result srv_check_health srv_check_state 
srv_agent_state bk_f_forced_id srv_f_forced_id srv_fqdn srv_port srvrecord

The last push seems to be running fine for now.

Willy

This email and any attachments are confidential and may also be privileged. If 
you are not the named recipient, please notify the sender immediately and do 
not disclose, use, store or copy the information contained herein. This is an 
email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 
118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees.


0001-BUILD-MINOR-51d-Updated-build-registration-output-to.patch
Description:  0001-BUILD-MINOR-51d-Updated-build-registration-output-to.patch

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

On Thu, Jun 13, 2019 at 03:21:36PM +, Ben Shillito wrote:
> Thanks both,
> 
> Ilya, I will take a look at that now.

I suspect it's totally unrelated, the failure was on this reg test and
may be just a race :

## Test case: reg-tests/checks/4be_1srv_health_checks.vtc ##
## test results in: 
"/tmp/haregtests-2019-06-13_15-05-08.R3rhNr/vtc.8210.558cb932"
 h11.5 CLI expect failed ~ "# be_id be_name srv_id srv_name srv_addr 
srv_op_state srv_admin_state srv_uweight srv_iweight srv_time_since_last_change 
srv_check_status srv_check_result srv_check_health srv_check_state 
srv_agent_state bk_f_forced_id srv_f_forced_id srv_fqdn srv_port srvrecord

The last push seems to be running fine for now.

Willy

RE: [PATCH] wurfl device detection build fixes and dummy library

[Ben Shillito]

Thanks both,

Ilya, I will take a look at that now.

Ben Shillito
Developer
[51Degrees]<https://51degrees.com/>
O: +44 1183 287152
E: b...@51degrees.com<mailto:b...@51degrees.com?subject=Your%20Email>
[https://51degrees.com/portals/0/images/twitterbird.png] 
@51Degrees<http://twitter.com/51Degreesmobi>
[https://51degrees.com/portals/0/images/linkedinicon.png] 
51Degrees<https://www.linkedin.com/company/2171864>
[Find out More]<https://51degrees.com/emailsig.aspx>

From: Илья Шипицин [mailto:chipits...@gmail.com]
Sent: 13 June 2019 16:19
To: Willy Tarreau 
Cc: Ben Shillito ; Christopher Faulet 
; HAProxy 
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library

Ben,

I enabled "trie" on one build configuration and ... it failed

https://travis-ci.com/haproxy/haproxy/jobs/207816969

it never failed before. can you have a look ?
(also, it did NOT fail in my own fork)

чт, 13 июн. 2019 г. в 20:03, Willy Tarreau mailto:w...@1wt.eu>>:
On Thu, Jun 13, 2019 at 08:01:14PM +0500,  ??? wrote:
> please find "travis-ci + 51degree" patch attached.

applied, thanks Ilya.
willy
This email and any attachments are confidential and may also be privileged. If 
you are not the named recipient, please notify the sender immediately and do 
not disclose, use, store or copy the information contained herein. This is an 
email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 
118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees.

Re: [PATCH] wurfl device detection build fixes and dummy library

[Илья Шипицин]

Ben,

I enabled "trie" on one build configuration and ... it failed

https://travis-ci.com/haproxy/haproxy/jobs/207816969

it never failed before. can you have a look ?
(also, it did NOT fail in my own fork)

чт, 13 июн. 2019 г. в 20:03, Willy Tarreau :

> On Thu, Jun 13, 2019 at 08:01:14PM +0500,  ??? wrote:
> > please find "travis-ci + 51degree" patch attached.
>
> applied, thanks Ilya.
> willy
>

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

On Thu, Jun 13, 2019 at 08:01:14PM +0500,  ??? wrote:
> please find "travis-ci + 51degree" patch attached.

applied, thanks Ilya.
willy

Re: [PATCH] wurfl device detection build fixes and dummy library

[Илья Шипицин]

please find "travis-ci + 51degree" patch attached.

чт, 13 июн. 2019 г. в 19:06, Willy Tarreau :

> On Thu, Jun 13, 2019 at 02:02:33PM +, Ben Shillito wrote:
> > Hi Willy,
> >
> > Yes, I agree the paths in the dummy library should match that of the
> actual library. And yes, that patch is good with me.
>
> Thanks for the fast response, now merged.
>
> Willy
>
From d2d22f75c1c45e84d3622e0cf45e48fc784b6ab8 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Thu, 13 Jun 2019 20:00:05 +0500
Subject: [PATCH] BUILD: travis-ci: add 51Degree device detection, update
 openssl to 1.1.1c

---
 .travis.yml | 13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 032ee13c..cfa68a93 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -5,10 +5,11 @@ language: c
 
 env:
   global:
-- FLAGS="USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_GETADDRINFO=1 USE_TFO=1 USE_SYSTEMD=1 USE_WURFL=1 WURFL_INC=contrib/wurfl WURFL_LIB=contrib/wurfl USE_DEVICEATLAS=1 DEVICEATLAS_SRC=contrib/deviceatlas"
+- FLAGS="USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_GETADDRINFO=1 USE_TFO=1 USE_SYSTEMD=1 USE_WURFL=1 WURFL_INC=contrib/wurfl WURFL_LIB=contrib/wurfl USE_DEVICEATLAS=1 DEVICEATLAS_SRC=contrib/deviceatlas USE_51DEGREES=1"
 - SSL_LIB=${HOME}/opt/lib
 - SSL_INC=${HOME}/opt/include
 - TMPDIR=/tmp
+- FIFTYONEDEGREES_SRC="contrib/51d/src/pattern"
 
 addons:
   apt:
@@ -23,13 +24,13 @@ matrix:
   include:
   - os: linux
 compiler: gcc
-env: TARGET=linux2628 FLAGS=
+env: TARGET=linux2628 FIFTYONEDEGREES_SRC="contrib/51d/src/trie"
   - os: linux-ppc64le
 compiler: gcc
-env: TARGET=linux2628 OPENSSL_VERSION=1.1.1b LABEL="linux-ppc64le"
+env: TARGET=linux2628 OPENSSL_VERSION=1.1.1c LABEL="linux-ppc64le"
   - os: linux
 compiler: clang
-env: TARGET=linux2628 OPENSSL_VERSION=1.1.1b
+env: TARGET=linux2628 OPENSSL_VERSION=1.1.1c
   - os: linux
 compiler: clang
 env: TARGET=linux2628 OPENSSL_VERSION=1.1.0j
@@ -53,7 +54,7 @@ matrix:
 env: TARGET=linux2628 FLAGS=
   - os: osx
 compiler: clang
-env: TARGET=osx FLAGS="USE_OPENSSL=1" OPENSSL_VERSION=1.1.1b
+env: TARGET=osx FLAGS="USE_OPENSSL=1" OPENSSL_VERSION=1.1.1c
   - os: windows
 install:
   - choco install bash make libssl-devel cygwin-devel gcc-core libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin
@@ -69,7 +70,7 @@ install:
 script:
   - if [ "${CC}"  = "clang" ]; then export FLAGS="$FLAGS USE_OBSOLETE_LINKER=1" DEBUG_CFLAGS="-g -fsanitize=address" LDFLAGS="-fsanitize=address"; fi
   - make -C contrib/wurfl
-  - make -j3 CC=$CC V=1 TARGET=$TARGET $FLAGS DEBUG_CFLAGS="$DEBUG_CFLAGS" LDFLAGS="$LDFLAGS" EXTRA_OBJS="$EXTRA_OBJS"
+  - make -j3 CC=$CC V=1 TARGET=$TARGET $FLAGS DEBUG_CFLAGS="$DEBUG_CFLAGS" LDFLAGS="$LDFLAGS" 51DEGREES_SRC="$FIFTYONEDEGREES_SRC" EXTRA_OBJS="$EXTRA_OBJS"
   - if [ "${TRAVIS_OS_NAME}" = "linux" ]; then export LD_LIBRARY_PATH="${HOME}/opt/lib:${LD_LIBRARY_PATH:-}"; fi
   - if [ "${TRAVIS_OS_NAME}" = "osx" ]; then export DYLD_LIBRARY_PATH="${HOME}/opt/lib:${DYLD_LIBRARY_PATH:-}"; fi
   - ./haproxy -vv
-- 
2.20.1

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

On Thu, Jun 13, 2019 at 02:02:33PM +, Ben Shillito wrote:
> Hi Willy,
> 
> Yes, I agree the paths in the dummy library should match that of the actual 
> library. And yes, that patch is good with me.

Thanks for the fast response, now merged.

Willy

RE: [PATCH] wurfl device detection build fixes and dummy library

[Ben Shillito]

Hi Willy,

Yes, I agree the paths in the dummy library should match that of the actual 
library. And yes, that patch is good with me.

Thanks,

Ben Shillito
Developer
O: +44 1183 287152
E: b...@51degrees.com
T: @51Degrees

-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu]
Sent: 13 June 2019 14:59
To: Ben Shillito 
Cc:  ??? ; Christopher Faulet 
; HAProxy 
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library

Ben,

what do you think of this one ?

Willy
This email and any attachments are confidential and may also be privileged. If 
you are not the named recipient, please notify the sender immediately and do 
not disclose, use, store or copy the information contained herein. This is an 
email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 
118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees.

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

Ben,

what do you think of this one ?

Willy
commit 5e4c5003c5cd5709e599aedbfdd5ef223dd3dc79
Author: Willy Tarreau 
Date:   Thu Jun 13 15:56:10 2019 +0200

CLEANUP: 51d: move the 51d dummy lib to contrib/51d/src to match the real 
lib

This way the directory structure remains the same as with the real lib and
one can apply the same build options regardless of where the lib is stored,
removing any possible confusion.

diff --git a/contrib/51d/cityhash/city.c b/contrib/51d/src/cityhash/city.c
similarity index 100%
rename from contrib/51d/cityhash/city.c
rename to contrib/51d/src/cityhash/city.c
diff --git a/contrib/51d/pattern/51Degrees.c 
b/contrib/51d/src/pattern/51Degrees.c
similarity index 100%
rename from contrib/51d/pattern/51Degrees.c
rename to contrib/51d/src/pattern/51Degrees.c
diff --git a/contrib/51d/pattern/51Degrees.h 
b/contrib/51d/src/pattern/51Degrees.h
similarity index 100%
rename from contrib/51d/pattern/51Degrees.h
rename to contrib/51d/src/pattern/51Degrees.h
diff --git a/contrib/51d/threading.c b/contrib/51d/src/threading.c
similarity index 100%
rename from contrib/51d/threading.c
rename to contrib/51d/src/threading.c
diff --git a/contrib/51d/trie/51Degrees.c b/contrib/51d/src/trie/51Degrees.c
similarity index 100%
rename from contrib/51d/trie/51Degrees.c
rename to contrib/51d/src/trie/51Degrees.c
diff --git a/contrib/51d/trie/51Degrees.h b/contrib/51d/src/trie/51Degrees.h
similarity index 100%
rename from contrib/51d/trie/51Degrees.h
rename to contrib/51d/src/trie/51Degrees.h
diff --git a/doc/51Degrees-device-detection.txt 
b/doc/51Degrees-device-detection.txt
index 2d4679d35..f0349abaa 100644
--- a/doc/51Degrees-device-detection.txt
+++ b/doc/51Degrees-device-detection.txt
@@ -58,9 +58,11 @@ For HAProxy developers who need to verify that their changes 
didn't affect the
 directory. This does not function, but implements the API such that the
 51Degrees module can be used (but not return any meaningful information). To
 test either Pattern or Hash Trie, build with:
-$ make TARGET= USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/pattern
+
+$ make TARGET= USE_51DEGREES=1 
51DEGREES_SRC=contrib/51d/src/pattern
 or
-$ make TARGET= USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/trie
+$ make TARGET= USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/trie
+
 respectively.
 
 The configuration file needs to set the following parameters:

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

On Thu, Jun 13, 2019 at 12:59:33PM +, Ben Shillito wrote:
> Thanks for the link, that's useful. Looks like the path to the source is the 
> issue.
> 
> It should be "51DEGREES_SRC=contrib/51d/pattern" rather than 
> "51DEGREES_SRC=contrib/51d/src/pattern".

That's what I just noticed as well, since it's the dummy lib it's important
to use the build syntax related to the dummy lib, which doesn't contain the
"src" subcomponent. This just makes me think that we should probably create
this "src" subcomponent in contrib/51d so that the directory structure
matches what you have in the official lib, it will then simplify the build
process and documentation.

Willy

RE: [PATCH] wurfl device detection build fixes and dummy library

[Ben Shillito]

Thanks for the link, that’s useful. Looks like the path to the source is the 
issue.

It should be “51DEGREES_SRC=contrib/51d/pattern” rather than 
“51DEGREES_SRC=contrib/51d/src/pattern”.

On your performance point, the Pattern algorithm offers the match metrics and 
is a bit more thorough in looking for “closest” matches if the User-Agent is 
unknown. It is more than fast enough for the majority of users, however use 
cases like ad tech can receive billions of requests per day. For that reason, 
it is only the high volume users which choose to forgo the extra analysis that 
Pattern offers in order to make use of the efficiency of Trie.

Regards,

Ben Shillito
Developer
[51Degrees]<https://51degrees.com/>
O: +44 1183 287152
E: b...@51degrees.com<mailto:b...@51degrees.com?subject=Your%20Email>
[https://51degrees.com/portals/0/images/twitterbird.png] 
@51Degrees<http://twitter.com/51Degreesmobi>
[https://51degrees.com/portals/0/images/linkedinicon.png] 
51Degrees<https://www.linkedin.com/company/2171864>
[Find out More]<https://51degrees.com/emailsig.aspx>

From: Илья Шипицин [mailto:chipits...@gmail.com]
Sent: 13 June 2019 13:27
To: Ben Shillito 
Cc: Willy Tarreau ; Christopher Faulet ; 
HAProxy 
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library



чт, 13 июн. 2019 г. в 17:09, Ben Shillito 
mailto:b...@51degrees.com>>:
In Travis the contrib version would be fine. But if you are testing something 
which will later be deployed to production, I would suggest using cloning the 
actual GitHub repo instead to be safe.


this is how I enable it on travis-ci

https://github.com/chipitsine/haproxy/blob/master/.travis.yml#L8

(you may see USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/pattern )

here's build

https://travis-ci.com/chipitsine/haproxy/jobs/207722666#L306-L310




In answer to your other question, yes, Pattern and Trie in HAProxy are mutually 
exclusive. Which one you choose depends on your needs. Trie provides the 
maximum performance, and Pattern takes slightly more time in order to give you 
better metrics indicating the quality of match. Unless you are handling upwards 
of a million requests per second, Pattern will be enough.

For more details on the differences between Pattern and Trie you can take a 
look at our “how device detection works” page here: 
https://51degrees.com/support/documentation/how-device-detection-works

ok, we'll test both

not clear, why one might need "maximum performance" algorithm and ... "not so 
fast another algorithm"
from my point of view, I would use the fastest algorithm all the time



And for the performance figures you can expect to see from both in HAProxy, see 
our benchmarks page here: 
https://51degrees.com/Support/Documentation/APIs/C-V32/Benchmarks

If you need any more information, or help setting up, do let me know.

Regards,

Ben Shillito
Developer
[Image removed by sender. 51Degrees]<https://51degrees.com/>
O: +44 1183 287152
E: b...@51degrees.com<mailto:b...@51degrees.com?subject=Your%20Email>
[Image removed by sender.] @51Degrees<http://twitter.com/51Degreesmobi>
[Image removed by sender.] 51Degrees<https://www.linkedin.com/company/2171864>
[Image removed by sender. Find out More]<https://51degrees.com/emailsig.aspx>

From: Илья Шипицин [mailto:chipits...@gmail.com<mailto:chipits...@gmail.com>]
Sent: 13 June 2019 11:31
To: Ben Shillito mailto:b...@51degrees.com>>
Cc: Willy Tarreau mailto:w...@1wt.eu>>; Christopher Faulet 
mailto:cfau...@haproxy.com>>; HAProxy 
mailto:haproxy@formilux.org>>
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library



чт, 13 июн. 2019 г. в 15:25, Ben Shillito 
mailto:b...@51degrees.com>>:
Hi,

The docs are correct. However, the 51Degrees source in the “contrib” directory 
should only be used for testing changes to the HAProxy source. The code 
contained in here does not contain any 51Degrees functionality, just method 
stubs to enable compilation and testing.

it's what I meant actually.
I added "USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/pattern" to travis-ci 
job in order to see whether it will build or not.

or do I need to clone github repo anyway ?


another question is "what should be added to travis-ci?" I see "src/pattern" 
and "src/trie" alternatives. Are they mutually exclusiive ? So, should we test 
either of them ?


You should use the 51Degrees source located at 
https://github.com/51degrees/device-detection.git. Then point 51DEGREES_SRC to 
either device-detection/src/pattern or device-detection/src/trie.

Perhaps if the documentation is not clear enough we should modify it, or even 
add a warning to the compilation.

Let me know if this gets you up and running.

Regards,

Ben Shillito
Developer
[Image removed by sender. 51Degrees]<https://51degrees.com/>
O: +44 1183 287152
E: b...@51degree

Re: [PATCH] wurfl device detection build fixes and dummy library

[Илья Шипицин]

чт, 13 июн. 2019 г. в 17:09, Ben Shillito :

> In Travis the contrib version would be fine. But if you are testing
> something which will later be deployed to production, I would suggest using
> cloning the actual GitHub repo instead to be safe.
>


this is how I enable it on travis-ci

https://github.com/chipitsine/haproxy/blob/master/.travis.yml#L8

(you may see USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/pattern )

here's build

https://travis-ci.com/chipitsine/haproxy/jobs/207722666#L306-L310




>
>
> In answer to your other question, yes, Pattern and Trie in HAProxy are
> mutually exclusive. Which one you choose depends on your needs. Trie
> provides the maximum performance, and Pattern takes slightly more time in
> order to give you better metrics indicating the quality of match. Unless
> you are handling upwards of a million requests per second, Pattern will be
> enough.
>
>
>
> For more details on the differences between Pattern and Trie you can take
> a look at our “how device detection works” page here:
> https://51degrees.com/support/documentation/how-device-detection-works
>

ok, we'll test both

not clear, why one might need "maximum performance" algorithm and ... "not
so fast another algorithm"
from my point of view, I would use the fastest algorithm all the time



>
>
> And for the performance figures you can expect to see from both in
> HAProxy, see our benchmarks page here:
> https://51degrees.com/Support/Documentation/APIs/C-V32/Benchmarks
>
>
>
> If you need any more information, or help setting up, do let me know.
>
>
>
> Regards,
>
>
>
> Ben Shillito
> Developer
>
> [image: 51Degrees] <https://51degrees.com/>
>
> O: +44 1183 287152 
> E: b...@51degrees.com 
>  @51Degrees <http://twitter.com/51Degreesmobi>
>  51Degrees <https://www.linkedin.com/company/2171864>
>
> [image: Find out More] <https://51degrees.com/emailsig.aspx>
>
>
>
> *From:* Илья Шипицин [mailto:chipits...@gmail.com]
> *Sent:* 13 June 2019 11:31
> *To:* Ben Shillito 
> *Cc:* Willy Tarreau ; Christopher Faulet ;
> HAProxy 
> *Subject:* Re: [PATCH] wurfl device detection build fixes and dummy
> library
>
>
>
>
>
>
>
> чт, 13 июн. 2019 г. в 15:25, Ben Shillito :
>
> Hi,
>
>
>
> The docs are correct. However, the 51Degrees source in the “contrib”
> directory should only be used for testing changes to the HAProxy source.
> The code contained in here does not contain any 51Degrees functionality,
> just method stubs to enable compilation and testing.
>
>
>
> it's what I meant actually.
>
> I added "USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/pattern" to
> travis-ci job in order to see whether it will build or not.
>
>
>
> or do I need to clone github repo anyway ?
>
>
>
>
>
> another question is "what should be added to travis-ci?" I see
> "src/pattern" and "src/trie" alternatives. Are they mutually exclusiive ?
> So, should we test either of them ?
>
>
>
>
>
> You should use the 51Degrees source located at
> https://github.com/51degrees/device-detection.git. Then point
> 51DEGREES_SRC to either device-detection/src/pattern or
> device-detection/src/trie.
>
>
>
> Perhaps if the documentation is not clear enough we should modify it, or
> even add a warning to the compilation.
>
>
>
> Let me know if this gets you up and running.
>
>
>
> Regards,
>
>
>
> Ben Shillito
> Developer
>
> [image: Image removed by sender. 51Degrees] <https://51degrees.com/>
>
> O: +44 1183 287152 
> E: b...@51degrees.com 
> [image: Image removed by sender.] @51Degrees
> <http://twitter.com/51Degreesmobi>
> [image: Image removed by sender.] 51Degrees
> <https://www.linkedin.com/company/2171864>
>
> [image: Image removed by sender. Find out More]
> <https://51degrees.com/emailsig.aspx>
>
>
>
> *From:* Илья Шипицин [mailto:chipits...@gmail.com]
> *Sent:* 13 June 2019 11:07
> *To:* Ben Shillito 
> *Cc:* Willy Tarreau ; Christopher Faulet ;
> HAProxy 
> *Subject:* Re: [PATCH] wurfl device detection build fixes and dummy
> library
>
>
>
> Ben,
>
>
>
> what is the proper way of building 51degree ?
>
>
>
> I added "USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/pattern" (as I seen
> in documentation)
>
>
>
> gcc -Iinclude -Iebtree -Wall -Wextra  -O2  -fno-strict-aliasing 
> -Wdeclaration-after-statement -fwrapv   -Wno-unused-label -Wno-sign-compare 
> -Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers 
> -Wno-clobbered -Wno-missing-field-initial

RE: [PATCH] wurfl device detection build fixes and dummy library

[Ben Shillito]

In Travis the contrib version would be fine. But if you are testing something 
which will later be deployed to production, I would suggest using cloning the 
actual GitHub repo instead to be safe.

In answer to your other question, yes, Pattern and Trie in HAProxy are mutually 
exclusive. Which one you choose depends on your needs. Trie provides the 
maximum performance, and Pattern takes slightly more time in order to give you 
better metrics indicating the quality of match. Unless you are handling upwards 
of a million requests per second, Pattern will be enough.

For more details on the differences between Pattern and Trie you can take a 
look at our “how device detection works” page here: 
https://51degrees.com/support/documentation/how-device-detection-works

And for the performance figures you can expect to see from both in HAProxy, see 
our benchmarks page here: 
https://51degrees.com/Support/Documentation/APIs/C-V32/Benchmarks

If you need any more information, or help setting up, do let me know.

Regards,

Ben Shillito
Developer
[51Degrees]<https://51degrees.com/>
O: +44 1183 287152
E: b...@51degrees.com<mailto:b...@51degrees.com?subject=Your%20Email>
[https://51degrees.com/portals/0/images/twitterbird.png] 
@51Degrees<http://twitter.com/51Degreesmobi>
[https://51degrees.com/portals/0/images/linkedinicon.png] 
51Degrees<https://www.linkedin.com/company/2171864>
[Find out More]<https://51degrees.com/emailsig.aspx>

From: Илья Шипицин [mailto:chipits...@gmail.com]
Sent: 13 June 2019 11:31
To: Ben Shillito 
Cc: Willy Tarreau ; Christopher Faulet ; 
HAProxy 
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library



чт, 13 июн. 2019 г. в 15:25, Ben Shillito 
mailto:b...@51degrees.com>>:
Hi,

The docs are correct. However, the 51Degrees source in the “contrib” directory 
should only be used for testing changes to the HAProxy source. The code 
contained in here does not contain any 51Degrees functionality, just method 
stubs to enable compilation and testing.

it's what I meant actually.
I added "USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/pattern" to travis-ci 
job in order to see whether it will build or not.

or do I need to clone github repo anyway ?


another question is "what should be added to travis-ci?" I see "src/pattern" 
and "src/trie" alternatives. Are they mutually exclusiive ? So, should we test 
either of them ?


You should use the 51Degrees source located at 
https://github.com/51degrees/device-detection.git. Then point 51DEGREES_SRC to 
either device-detection/src/pattern or device-detection/src/trie.

Perhaps if the documentation is not clear enough we should modify it, or even 
add a warning to the compilation.

Let me know if this gets you up and running.

Regards,

Ben Shillito
Developer
[Image removed by sender. 51Degrees]<https://51degrees.com/>
O: +44 1183 287152
E: b...@51degrees.com<mailto:b...@51degrees.com?subject=Your%20Email>
[Image removed by sender.] @51Degrees<http://twitter.com/51Degreesmobi>
[Image removed by sender.] 51Degrees<https://www.linkedin.com/company/2171864>
[Image removed by sender. Find out More]<https://51degrees.com/emailsig.aspx>

From: Илья Шипицин [mailto:chipits...@gmail.com<mailto:chipits...@gmail.com>]
Sent: 13 June 2019 11:07
To: Ben Shillito mailto:b...@51degrees.com>>
Cc: Willy Tarreau mailto:w...@1wt.eu>>; Christopher Faulet 
mailto:cfau...@haproxy.com>>; HAProxy 
mailto:haproxy@formilux.org>>
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library

Ben,

what is the proper way of building 51degree ?

I added "USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/pattern" (as I seen in 
documentation)


gcc -Iinclude -Iebtree -Wall -Wextra  -O2  -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -Wno-unused-label -Wno-sign-compare 
-Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers 
-Wno-clobbered -Wno-missing-field-initializers-Wtype-limits   
-DUSE_EPOLL-DUSE_NETFILTER -DUSE_PCRE -DUSE_PCRE_JIT   -DUSE_POLL  
-DUSE_THREAD -DUSE_TPROXY -DUSE_LINUX_TPROXY -DUSE_LINUX_SPLICE 
-DUSE_LIBCRYPT -DUSE_CRYPT_H  -DUSE_GETADDRINFO -DUSE_OPENSSL -DUSE_LUA 
-DUSE_FUTEX -DUSE_ACCEPT4  -DUSE_ZLIB  -DUSE_CPU_AFFINITY -DUSE_TFO  -DUSE_DL 
-DUSE_RT -DUSE_DEVICEATLAS -DUSE_51DEGREES -DUSE_WURFL -DUSE_SYSTEMD  
-DUSE_PRCTL -DUSE_THREAD_DUMP   -I/home/travis/opt/include 
-I/usr/include/lua5.3 -I/usr/include/lua5.3 -Icontrib/deviceatlas 
-Icontrib/src/trie -Icontrib/wurfl -DUSE_PCRE -I/usr/include  
-DCONFIG_HAPROXY_VERSION=\"2.0-dev7\" -DCONFIG_HAPROXY_DATE=\"2019/06/11\" -c 
-o src/da.o src/da.c

make: *** No rule to make target 'contrib/src/trie/../cityhash/city.o', needed 
by 'haproxy'.  Stop.

make: *** Waiting for unfinished jobs

is documentation correct ? or I should use some different way to build

ср, 1

Re: [PATCH] wurfl device detection build fixes and dummy library

[Илья Шипицин]

чт, 13 июн. 2019 г. в 15:25, Ben Shillito :

> Hi,
>
>
>
> The docs are correct. However, the 51Degrees source in the “contrib”
> directory should only be used for testing changes to the HAProxy source.
> The code contained in here does not contain any 51Degrees functionality,
> just method stubs to enable compilation and testing.
>

it's what I meant actually.
I added "USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/pattern" to
travis-ci job in order to see whether it will build or not.

or do I need to clone github repo anyway ?


another question is "what should be added to travis-ci?" I see
"src/pattern" and "src/trie" alternatives. Are they mutually exclusiive ?
So, should we test either of them ?


>
>
> You should use the 51Degrees source located at
> https://github.com/51degrees/device-detection.git. Then point
> 51DEGREES_SRC to either device-detection/src/pattern or
> device-detection/src/trie.
>
>
>
> Perhaps if the documentation is not clear enough we should modify it, or
> even add a warning to the compilation.
>
>
>
> Let me know if this gets you up and running.
>
>
>
> Regards,
>
>
>
> Ben Shillito
> Developer
>
> [image: 51Degrees] <https://51degrees.com/>
>
> O: +44 1183 287152 
> E: b...@51degrees.com 
>  @51Degrees <http://twitter.com/51Degreesmobi>
>  51Degrees <https://www.linkedin.com/company/2171864>
>
> [image: Find out More] <https://51degrees.com/emailsig.aspx>
>
>
>
> *From:* Илья Шипицин [mailto:chipits...@gmail.com]
> *Sent:* 13 June 2019 11:07
> *To:* Ben Shillito 
> *Cc:* Willy Tarreau ; Christopher Faulet ;
> HAProxy 
> *Subject:* Re: [PATCH] wurfl device detection build fixes and dummy
> library
>
>
>
> Ben,
>
>
>
> what is the proper way of building 51degree ?
>
>
>
> I added "USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/pattern" (as I seen
> in documentation)
>
>
>
> gcc -Iinclude -Iebtree -Wall -Wextra  -O2  -fno-strict-aliasing 
> -Wdeclaration-after-statement -fwrapv   -Wno-unused-label -Wno-sign-compare 
> -Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers 
> -Wno-clobbered -Wno-missing-field-initializers-Wtype-limits   
> -DUSE_EPOLL-DUSE_NETFILTER -DUSE_PCRE -DUSE_PCRE_JIT   -DUSE_POLL  
> -DUSE_THREAD -DUSE_TPROXY -DUSE_LINUX_TPROXY -DUSE_LINUX_SPLICE 
> -DUSE_LIBCRYPT -DUSE_CRYPT_H  -DUSE_GETADDRINFO -DUSE_OPENSSL -DUSE_LUA 
> -DUSE_FUTEX -DUSE_ACCEPT4  -DUSE_ZLIB  -DUSE_CPU_AFFINITY -DUSE_TFO  -DUSE_DL 
> -DUSE_RT -DUSE_DEVICEATLAS -DUSE_51DEGREES -DUSE_WURFL -DUSE_SYSTEMD  
> -DUSE_PRCTL -DUSE_THREAD_DUMP   -I/home/travis/opt/include 
> -I/usr/include/lua5.3 -I/usr/include/lua5.3 -Icontrib/deviceatlas 
> -Icontrib/src/trie -Icontrib/wurfl -DUSE_PCRE -I/usr/include  
> -DCONFIG_HAPROXY_VERSION=\"2.0-dev7\" -DCONFIG_HAPROXY_DATE=\"2019/06/11\" -c 
> -o src/da.o src/da.c
>
> make: *** No rule to make target 'contrib/src/trie/../cityhash/city.o', 
> needed by 'haproxy'.  Stop.
>
> make: *** Waiting for unfinished jobs
>
> is documentation correct ? or I should use some different way to build
>
>
>
> ср, 12 июн. 2019 г. в 22:01, Ben Shillito :
>
> Hi Willy,
>
> Great, thanks for those changes, and good spot.
>
> I agree that this is a significant step forward, and having the entire
> codebase testable in CI will certainly make everything that bit smoother.
>
> Thanks,
>
> Ben Shillito
> Developer
> O: +44 1183 287152
> E: b...@51degrees.com
> T: @51Degrees
>
> -Original Message-
> From: Willy Tarreau [mailto:w...@1wt.eu]
> Sent: 12 June 2019 17:07
> To: Ben Shillito 
> Cc: Christopher Faulet ; HAProxy <
> haproxy@formilux.org>
> Subject: Re: [PATCH] wurfl device detection build fixes and dummy library
>
> On Wed, Jun 12, 2019 at 02:49:37PM +, Ben Shillito wrote:
> > While I was working on the HTX changes, I thought it was probably a
> > good time to also implement the dummy library as I had my brain in
> 'HAProxy mode'.
>
> Ah, excellent, thank you :
>
>   $./haproxy -vv|grep -i 51d
>   Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE
> -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED
> -REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE
> +LIBCRYPT +CRYPT_H -VSYSCALL -GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4
> -MY_ACCEPT4 -ZLIB +SLZ +CPU_AFFINITY -TFO +NS +DL +RT -DEVICEATLAS
> -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
>   Built with 51Degrees support.
>
> :-)
>
> Ilya will likely be happy to see that w

RE: [PATCH] wurfl device detection build fixes and dummy library

[Ben Shillito]

Hi,

The docs are correct. However, the 51Degrees source in the “contrib” directory 
should only be used for testing changes to the HAProxy source. The code 
contained in here does not contain any 51Degrees functionality, just method 
stubs to enable compilation and testing.

You should use the 51Degrees source located at 
https://github.com/51degrees/device-detection.git. Then point 51DEGREES_SRC to 
either device-detection/src/pattern or device-detection/src/trie.

Perhaps if the documentation is not clear enough we should modify it, or even 
add a warning to the compilation.

Let me know if this gets you up and running.

Regards,

Ben Shillito
Developer
[51Degrees]<https://51degrees.com/>
O: +44 1183 287152
E: b...@51degrees.com<mailto:b...@51degrees.com?subject=Your%20Email>
[https://51degrees.com/portals/0/images/twitterbird.png] 
@51Degrees<http://twitter.com/51Degreesmobi>
[https://51degrees.com/portals/0/images/linkedinicon.png] 
51Degrees<https://www.linkedin.com/company/2171864>
[Find out More]<https://51degrees.com/emailsig.aspx>

From: Илья Шипицин [mailto:chipits...@gmail.com]
Sent: 13 June 2019 11:07
To: Ben Shillito 
Cc: Willy Tarreau ; Christopher Faulet ; 
HAProxy 
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library

Ben,

what is the proper way of building 51degree ?

I added "USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/pattern" (as I seen in 
documentation)


gcc -Iinclude -Iebtree -Wall -Wextra  -O2  -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv   -Wno-unused-label -Wno-sign-compare 
-Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers 
-Wno-clobbered -Wno-missing-field-initializers-Wtype-limits   
-DUSE_EPOLL-DUSE_NETFILTER -DUSE_PCRE -DUSE_PCRE_JIT   -DUSE_POLL  
-DUSE_THREAD -DUSE_TPROXY -DUSE_LINUX_TPROXY -DUSE_LINUX_SPLICE 
-DUSE_LIBCRYPT -DUSE_CRYPT_H  -DUSE_GETADDRINFO -DUSE_OPENSSL -DUSE_LUA 
-DUSE_FUTEX -DUSE_ACCEPT4  -DUSE_ZLIB  -DUSE_CPU_AFFINITY -DUSE_TFO  -DUSE_DL 
-DUSE_RT -DUSE_DEVICEATLAS -DUSE_51DEGREES -DUSE_WURFL -DUSE_SYSTEMD  
-DUSE_PRCTL -DUSE_THREAD_DUMP   -I/home/travis/opt/include 
-I/usr/include/lua5.3 -I/usr/include/lua5.3 -Icontrib/deviceatlas 
-Icontrib/src/trie -Icontrib/wurfl -DUSE_PCRE -I/usr/include  
-DCONFIG_HAPROXY_VERSION=\"2.0-dev7\" -DCONFIG_HAPROXY_DATE=\"2019/06/11\" -c 
-o src/da.o src/da.c

make: *** No rule to make target 'contrib/src/trie/../cityhash/city.o', needed 
by 'haproxy'.  Stop.

make: *** Waiting for unfinished jobs


is documentation correct ? or I should use some different way to build

ср, 12 июн. 2019 г. в 22:01, Ben Shillito 
mailto:b...@51degrees.com>>:
Hi Willy,

Great, thanks for those changes, and good spot.

I agree that this is a significant step forward, and having the entire codebase 
testable in CI will certainly make everything that bit smoother.

Thanks,

Ben Shillito
Developer
O: +44 1183 287152
E: b...@51degrees.com<mailto:b...@51degrees.com>
T: @51Degrees

-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu<mailto:w...@1wt.eu>]
Sent: 12 June 2019 17:07
To: Ben Shillito mailto:b...@51degrees.com>>
Cc: Christopher Faulet mailto:cfau...@haproxy.com>>; 
HAProxy mailto:haproxy@formilux.org>>
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library

On Wed, Jun 12, 2019 at 02:49:37PM +, Ben Shillito wrote:
> While I was working on the HTX changes, I thought it was probably a
> good time to also implement the dummy library as I had my brain in 'HAProxy 
> mode'.

Ah, excellent, thank you :

  $./haproxy -vv|grep -i 51d
  Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE -PCRE_JIT 
-PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED -REGPARM 
-STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT 
+CRYPT_H -VSYSCALL -GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 -ZLIB 
+SLZ +CPU_AFFINITY -TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD 
-OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
  Built with 51Degrees support.

:-)

Ilya will likely be happy to see that we can now build 100% of our codebase in 
the CI, this is a significant step forward!

I performed very minor changes to your patches, for the first one I marked it 
"BUG/MINOR" so that we backport it to 1.9 (since it's still broken there) and 
for the second I fixed the doc where you accidently dropped "51d" after 
"contrib/" in the build command :-)

Thanks a lot for your responsiveness,
Willy
This email and any attachments are confidential and may also be privileged. If 
you are not the named recipient, please notify the sender immediately and do 
not disclose, use, store or copy the information contained herein. This is an 
email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 
118 328 7152; E: i...@51degrees.com<mailto:i...@5

Re: [PATCH] wurfl device detection build fixes and dummy library

[Илья Шипицин]

Ben,

what is the proper way of building 51degree ?

I added "USE_51DEGREES=1 51DEGREES_SRC=contrib/51d/src/pattern" (as I seen
in documentation)

gcc -Iinclude -Iebtree -Wall -Wextra  -O2  -fno-strict-aliasing
-Wdeclaration-after-statement -fwrapv   -Wno-unused-label
-Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration
-Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers
   -Wtype-limits   -DUSE_EPOLL-DUSE_NETFILTER -DUSE_PCRE
-DUSE_PCRE_JIT   -DUSE_POLL  -DUSE_THREAD -DUSE_TPROXY
-DUSE_LINUX_TPROXY -DUSE_LINUX_SPLICE -DUSE_LIBCRYPT -DUSE_CRYPT_H
-DUSE_GETADDRINFO -DUSE_OPENSSL -DUSE_LUA -DUSE_FUTEX -DUSE_ACCEPT4
-DUSE_ZLIB  -DUSE_CPU_AFFINITY -DUSE_TFO  -DUSE_DL -DUSE_RT
-DUSE_DEVICEATLAS -DUSE_51DEGREES -DUSE_WURFL -DUSE_SYSTEMD
-DUSE_PRCTL -DUSE_THREAD_DUMP   -I/home/travis/opt/include
-I/usr/include/lua5.3 -I/usr/include/lua5.3 -Icontrib/deviceatlas
-Icontrib/src/trie -Icontrib/wurfl -DUSE_PCRE -I/usr/include
-DCONFIG_HAPROXY_VERSION=\"2.0-dev7\"
-DCONFIG_HAPROXY_DATE=\"2019/06/11\" -c -o src/da.o src/da.c
make: *** No rule to make target
'contrib/src/trie/../cityhash/city.o', needed by 'haproxy'.  Stop.
make: *** Waiting for unfinished jobs


is documentation correct ? or I should use some different way to build


ср, 12 июн. 2019 г. в 22:01, Ben Shillito :

> Hi Willy,
>
> Great, thanks for those changes, and good spot.
>
> I agree that this is a significant step forward, and having the entire
> codebase testable in CI will certainly make everything that bit smoother.
>
> Thanks,
>
> Ben Shillito
> Developer
> O: +44 1183 287152
> E: b...@51degrees.com
> T: @51Degrees
>
> -Original Message-
> From: Willy Tarreau [mailto:w...@1wt.eu]
> Sent: 12 June 2019 17:07
> To: Ben Shillito 
> Cc: Christopher Faulet ; HAProxy <
> haproxy@formilux.org>
> Subject: Re: [PATCH] wurfl device detection build fixes and dummy library
>
> On Wed, Jun 12, 2019 at 02:49:37PM +, Ben Shillito wrote:
> > While I was working on the HTX changes, I thought it was probably a
> > good time to also implement the dummy library as I had my brain in
> 'HAProxy mode'.
>
> Ah, excellent, thank you :
>
>   $./haproxy -vv|grep -i 51d
>   Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE
> -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED
> -REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE
> +LIBCRYPT +CRYPT_H -VSYSCALL -GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4
> -MY_ACCEPT4 -ZLIB +SLZ +CPU_AFFINITY -TFO +NS +DL +RT -DEVICEATLAS
> -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
>   Built with 51Degrees support.
>
> :-)
>
> Ilya will likely be happy to see that we can now build 100% of our
> codebase in the CI, this is a significant step forward!
>
> I performed very minor changes to your patches, for the first one I marked
> it "BUG/MINOR" so that we backport it to 1.9 (since it's still broken
> there) and for the second I fixed the doc where you accidently dropped
> "51d" after "contrib/" in the build command :-)
>
> Thanks a lot for your responsiveness,
> Willy
> This email and any attachments are confidential and may also be
> privileged. If you are not the named recipient, please notify the sender
> immediately and do not disclose, use, store or copy the information
> contained herein. This is an email from 51Degrees.mobi Limited, 5 Charlotte
> Close, Reading. RG47BY. T: +44 118 328 7152; E: i...@51degrees.com;
> 51Degrees.mobi Limited t/as 51Degrees.
>
>

RE: [PATCH] wurfl device detection build fixes and dummy library

[Ben Shillito]

Hi Willy,

Great, thanks for those changes, and good spot.

I agree that this is a significant step forward, and having the entire codebase 
testable in CI will certainly make everything that bit smoother.

Thanks,

Ben Shillito
Developer
O: +44 1183 287152
E: b...@51degrees.com
T: @51Degrees

-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu]
Sent: 12 June 2019 17:07
To: Ben Shillito 
Cc: Christopher Faulet ; HAProxy 
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library

On Wed, Jun 12, 2019 at 02:49:37PM +, Ben Shillito wrote:
> While I was working on the HTX changes, I thought it was probably a
> good time to also implement the dummy library as I had my brain in 'HAProxy 
> mode'.

Ah, excellent, thank you :

  $./haproxy -vv|grep -i 51d
  Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE -PCRE_JIT 
-PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED -REGPARM 
-STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT 
+CRYPT_H -VSYSCALL -GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 -ZLIB 
+SLZ +CPU_AFFINITY -TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD 
-OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
  Built with 51Degrees support.

:-)

Ilya will likely be happy to see that we can now build 100% of our codebase in 
the CI, this is a significant step forward!

I performed very minor changes to your patches, for the first one I marked it 
"BUG/MINOR" so that we backport it to 1.9 (since it's still broken there) and 
for the second I fixed the doc where you accidently dropped "51d" after 
"contrib/" in the build command :-)

Thanks a lot for your responsiveness,
Willy
This email and any attachments are confidential and may also be privileged. If 
you are not the named recipient, please notify the sender immediately and do 
not disclose, use, store or copy the information contained herein. This is an 
email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 
118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees.

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

On Wed, Jun 12, 2019 at 02:49:37PM +, Ben Shillito wrote:
> While I was working on the HTX changes, I thought it was probably a good time
> to also implement the dummy library as I had my brain in 'HAProxy mode'.

Ah, excellent, thank you :

  $./haproxy -vv|grep -i 51d
  Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE -PCRE_JIT 
-PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED -REGPARM 
-STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT 
+CRYPT_H -VSYSCALL -GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 -ZLIB 
+SLZ +CPU_AFFINITY -TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD 
-OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
  Built with 51Degrees support.

:-)

Ilya will likely be happy to see that we can now build 100% of our
codebase in the CI, this is a significant step forward!

I performed very minor changes to your patches, for the first one I marked
it "BUG/MINOR" so that we backport it to 1.9 (since it's still broken there)
and for the second I fixed the doc where you accidently dropped "51d" after
"contrib/" in the build command :-)

Thanks a lot for your responsiveness,
Willy

RE: [PATCH] wurfl device detection build fixes and dummy library

[Ben Shillito]

Hi Willy,

No worries, that's fine.

While I was working on the HTX changes, I thought it was probably a good time 
to also implement the dummy library as I had my brain in 'HAProxy mode'.

Both patches are attached ready to be merged.

If there is anything missing, let me know and can I'll try to make sure you 
have everything you need for a weekend release.

Thanks,

Ben Shillito
Developer
O: +44 1183 287152
E: b...@51degrees.com
T: @51Degrees

-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu]
Sent: 12 June 2019 10:12
To: Ben Shillito 
Cc: Christopher Faulet ; HAProxy 
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library

On Wed, Jun 12, 2019 at 08:52:53AM +, Ben Shillito wrote:
> Hi Willy,
>
> This unfortunately fell down our list of priorities in the last few weeks.

Oh I certainly can understand, and I'm sorry I forgot about it so I didn't ping 
you earlier.

> However, as this is a bit more urgent now with your weekend release, I
> will get the change for HTX awareness to you either today or tomorrow
> if that is ok with you?

OK that's perfect, thank you! It will be less work on my side to merge your fix 
than to write the verification code to refuse to start!

Thanks,
Willy
This email and any attachments are confidential and may also be privileged. If 
you are not the named recipient, please notify the sender immediately and do 
not disclose, use, store or copy the information contained herein. This is an 
email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 
118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees.


0001-FEAT-51d-htx-The-_51d_fetch-method-and-the-methods-i.patch
Description:  0001-FEAT-51d-htx-The-_51d_fetch-method-and-the-methods-i.patch


0002-MINOR-51d-Added-dummy-libraries-for-the-51Degrees-mo.patch
Description:  0002-MINOR-51d-Added-dummy-libraries-for-the-51Degrees-mo.patch

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

On Wed, Jun 12, 2019 at 08:52:53AM +, Ben Shillito wrote:
> Hi Willy,
> 
> This unfortunately fell down our list of priorities in the last few weeks.

Oh I certainly can understand, and I'm sorry I forgot about it so I didn't
ping you earlier.

> However, as this is a bit more urgent now with your weekend release, I will
> get the change for HTX awareness to you either today or tomorrow if that is
> ok with you?

OK that's perfect, thank you! It will be less work on my side to merge your
fix than to write the verification code to refuse to start!

Thanks,
Willy

RE: [PATCH] wurfl device detection build fixes and dummy library

[Ben Shillito]

Hi Willy,

This unfortunately fell down our list of priorities in the last few weeks. 
However, as this is a bit more urgent now with your weekend release, I will get 
the change for HTX awareness to you either today or tomorrow if that is ok with 
you?

Thanks,

Ben Shillito
Developer
O: +44 1183 287152
E: b...@51degrees.com
T: @51Degrees
-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu]
Sent: 12 June 2019 09:14
To: Ben Shillito 
Cc: Christopher Faulet ; HAProxy 
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library

Hi Ben,

On Wed, Apr 24, 2019 at 06:20:16PM +0200, Christopher Faulet wrote:
> Le 24/04/2019 à 17:56, Ben Shillito a écrit :
> > Hi Willy,
> >
> > Thanks for the update. We will take a look and get a patch over to you.
> >
>
> Hi,
>
> Ben, the function _51d_fetch() is not HTX aware. Take a look at other
> HTTP sample fetches in src/http_fetch.c.

I'm just realizing that we never got your patch for this. Do you have a patch 
handy or should we currently write a quick one to prevent one from starting 
with both 51d and HTX enabled ? HTX is on by default in
2.0 and the final release is expected around this week-end...

Please just let me know, at least I don't want users to meet random crashes 
when using this code.

Thanks,
Willy
This email and any attachments are confidential and may also be privileged. If 
you are not the named recipient, please notify the sender immediately and do 
not disclose, use, store or copy the information contained herein. This is an 
email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 
118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees.

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

Hi Ben,

On Wed, Apr 24, 2019 at 06:20:16PM +0200, Christopher Faulet wrote:
> Le 24/04/2019 à 17:56, Ben Shillito a écrit :
> > Hi Willy,
> > 
> > Thanks for the update. We will take a look and get a patch over to you.
> > 
> 
> Hi,
> 
> Ben, the function _51d_fetch() is not HTX aware. Take
> a look at other HTTP sample fetches in src/http_fetch.c.

I'm just realizing that we never got your patch for this. Do you have
a patch handy or should we currently write a quick one to prevent one
from starting with both 51d and HTX enabled ? HTX is on by default in
2.0 and the final release is expected around this week-end...

Please just let me know, at least I don't want users to meet random
crashes when using this code.

Thanks,
Willy

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

Hi Max,

I'll respond on some points here.

On Thu, May 09, 2019 at 06:03:58PM +0200, Massimiliano Bellomi wrote:
> Hi Christopher,
> 
> here Massimiliano, from Scientiamobile Engineering team.
> 
> We started working on your suggestions.
> 
> Doing this, I noticed that *send_log()* seems not working if called inside
> module's init function.

Indeed. There's a historical distinction between logs (which are only for
runtime traffic) and alerts/warnings that are only for startup (typically
config errors). So while you should use send_log() and friends to report
critical events (it's already done for you at the end of the stream for
regular activity), you should only use ha_warning() to report startup
warnings that can be avoided by a functionally equivalent configuration,
and ha_alert() to report alerts corresponding to a refusal to start.
Typically you'll use a warning to indicate that an old keyword is
deprecated and is supposed to be replaced by something else but that it
was still done for the user, and you'll use an alert for something wrong,
that makes no sense or that cannot work.

> Has anything changed in the latest version of HA (in 1.7 that works) that
> we need to pay attention to ?

I find it strange that logs used to work upon startup in 1.7 or that
could have been just an accident, such as a startup race depending on
the declaration order, since logs were quite simpler by then and
probably didn't require as much initialization, so maybe the socket
was initialized during parsing and everything was immediately usable.

Hoping this helps,
Willy

Re: [PATCH] wurfl device detection build fixes and dummy library

[Massimiliano Bellomi]

Hi Christopher,

here Massimiliano, from Scientiamobile Engineering team.

We started working on your suggestions.

Doing this, I noticed that *send_log()* seems not working if called inside
module's init function.

e.g.
*send_log(NULL, LOG_NOTICE, "WURFL: Loading module v.%s\n",
HA_WURFL_MODULE_VERSION);*
doesn't produce any syslog message, while if called inside our fetch
functions, it works perfectly.

Digging quicky into this, the reason is that *send_log()* early returns
because of logline==NULL (when called inside init)
Are we doing something wrong ?
Has anything changed in the latest version of HA (in 1.7 that works) that
we need to pay attention to ?

Thank you in advance for any suggestion

Regards,
-Max

On Wed, Apr 24, 2019 at 6:11 PM Christopher Faulet 
wrote:

> Le 23/04/2019 à 11:10, Willy Tarreau a écrit :
> > Hi Paul,
> >
> > On Fri, Apr 19, 2019 at 06:45:22PM +0200, Paul Stephen Borile wrote:
> >> Hi Willy,
> >>
> >> fine for me, thanks for the adjustments and no problem backporting this
> to
> >> 1.9.
> >> I also confirm that the contact email address is working correctly.
> >
> > Fine thank you. I could finish the polishing (add USE_WURFL to the list
> > of known and reported build options in the makefile) and I've
> reintegrated
> > the code now.
> >
> > You probably don't see the value in having the dummy library, but for
> > developers it's invaluable. I have now updated my build script to build
> > with USE_WURFL=1 by default so that I'll now see if anything causes
> > warnings or errors there if we touch structures that are used by your
> > code.
> >
> > It would be really awesome if Device Atlas and 51Degrees could do the
> > same, as the build coverage becomes much better with very little effort
> > for everyone. David, Ben, if you read this, please have a look at
> > contrib/wurfl to get an idea of what is sufficient to have your code
> > always built by anyone. Patches welcome :-)
> >
> Hi Paul,
>
> I quickly reviewed the wurfl integration. I tested it with the dummy
> library. It really made my tests easier, many thanks.
>
> First, I have a segfault when I use the sample fetch "wurfl-get" at line
> 521. It happens when I try to retrieve an unknown data (I mean not
> listed in wurfl-information-list). Here is my config:
>
>  global
>  ...
>  wurfl-data-file /usr/share/wurfl/wurfl.zip
>  wurfl-information-list wurfl_id model_name
>  ...
>
>  frontend http
>  ...
>  http-request set-header X-WURFL-Properties
> %[wurfl-get(wurfl_id,is_tablet)] # is_tablet is not in the list
>  
>
> Then, at the beginning of the wurfl sample fetches, the channel validity
> must be checked calling the macro CHECK_HTTP_MESSAGE_FIRST(). Otherwise,
> some processing can be performed on empty buffers or uninitialized data.
>
> Finally, the function ha_wurfl_retrieve_header() is not HTX aware. Take
> a look at others HTTP sample fetches in src/http_fetch.c.
>
> Just a suggestion. It could be cool to call it from the dummy library,
> in wurfl_lookup(). This way, we will be able to test this part.
>
> Regards,
> --
> Christopher Faulet
>
>

-- 
Massimiliano Bellomi
Senior Software Engineer
Scientiamobile Italy -  massimili...@scientiamobile.com +39 338 6990288
Milano Office : +39 02 620227260
skype: massimiliano.bellomi

Re: [PATCH] wurfl device detection build fixes and dummy library

[Aaron Park (Support)]

# Please type your reply above this line #

You are registered as a cc on this help desk request and are thus receiving 
email notifications on all updates to the request.
Reply to this email to add a comment to the request.

--

Aaron Park, May 7, 15:30 EDT

Hi Willy,

I just wanted to check in to let you know that our engineers are continuing to 
build a patch to address the varying issues you are seeing. 

Because this build will take some time, unless you have any other questions, we 
will close out this ticket for now and will keep you informed of any updates we 
have for you.

Thanks,

Aaron


e: supp...@scientiamobile.com
ScientiaMobile Customer Support Team

--

Willy Tarreau, Apr 19, 10:49 EDT

Sorry, with the patches this time.

Willy

Attachment(s):
0007-WIP-wurfl-pass-fPIC-when-compiling.patch - 
https://support.scientiamobile.com/attachments/token/fAR7m6yvCQlVHA4dqp23I5sJS/?name=0007-WIP-wurfl-pass-fPIC-when-compiling.patch
0008-WIP-wurfl-fix-broken-symlinks.patch - 
https://support.scientiamobile.com/attachments/token/2Nrk5JLukGKWfCDj6rXLNFeik/?name=0008-WIP-wurfl-fix-broken-symlinks.patch
0009-WIP-wurfl-address-build-issues-by-doing-a-static-lib.patch - 
https://support.scientiamobile.com/attachments/token/mJXKrE2ecZHS7RjwXwTwbbQHi/?name=0009-WIP-wurfl-address-build-issues-by-doing-a-static-lib.patch
0010-WIP-wurfl-indicate-in-haproxy-vv-the-wurfl-version-i.patch - 
https://support.scientiamobile.com/attachments/token/Pc7IJxxno71IOlfHSByian8dp/?name=0010-WIP-wurfl-indicate-in-haproxy-vv-the-wurfl-version-i.patch
0011-WIP-wurfl-move-wurfl.h-into-wurfl-to-maintain-direct.patch - 
https://support.scientiamobile.com/attachments/token/qkvpDMK65KKVXVJsJyrNSxp7t/?name=0011-WIP-wurfl-move-wurfl.h-into-wurfl-to-maintain-direct.patch
0012-WIP-wurfl-mention-how-to-build-the-dummy-lib-in-the-.patch - 
https://support.scientiamobile.com/attachments/token/IWptQoEt33SpfGBPLJWNh5ZNL/?name=0012-WIP-wurfl-mention-how-to-build-the-dummy-lib-in-the-.patch
0013-WIP-wurfl-rename-makefile-to-Makefile.patch - 
https://support.scientiamobile.com/attachments/token/BD9188uXGpWCBfCxcungaYVoJ/?name=0013-WIP-wurfl-rename-makefile-to-Makefile.patch

--

Willy Tarreau, Apr 19, 10:46 EDT

Hi Paul,

On Thu, Apr 18, 2019 at 02:46:17PM +0200, Paul Stephen Borile wrote:
> please find attached to this email the 6 patches that cover various areas
> of restyling of
> the WURFL device detection feature for HAProxy. All patches can be back
> ported to 1.9 if necessary.
> Last patch is a dummy WURFL library that can be used to build/run haproxy
> compiled with the USE_WURFL option to make easier checking for any build
> problem in the future.
> We'll try to do the same and make sure that the module does not break
> builds again as happened in the past.

So I gave a look to this patch set and had to perform a few adjustments
to make it work but now it looks OK. I'm attaching the changes I made so
that you can review them, they're all related to the dummy lib in order
to 1) fix its build and 2) ease the testing without having to modify the
build environment (since adding non-standard stuff into /usr/include or
/usr/lib is a no-go on most development environments).

I figured that it was much simpler to build a ".a" from the file so that
it can naturally be loaded by the regular build process. I added the
ability to report the libwurfl version in "haproxy -vv" and when the
dummy lib is detected, it's explicitly mentioned "dummy library" there
so that you don't have to deal with false positives when users report
issues. I also added a little bit of doc explaining to haproxy devs
how to build with wurfl. This way I think it could be added by default
to any developer's build script so that it never breaks in the future.

I'm attaching my changes. I'm fine with retrofitting them into your
patches if they look OK to you. Please just let me know if you're OK to
go with this (and if you're OK with me backporting this to 1.9 so that
we can fix 1.9 once for all).

Thanks!
Willy

PS: I've CCed the contact address in the maintainers file just to verify
that there is no typo there, please confirm that it was properly
received.'


This email is a service from ScientiaMobile.









[PM6MEL-7Z9M]

Re: [PATCH] wurfl device detection build fixes and dummy library

[Paul Stephen Borile]

Hi Christopher,

thanks for all suggestions : I'll be working on a patch to address the
various issues.
Yes, the callback should be called (once with an existing header, once with
a non existing one) as that is code to be tested as well : I'll include
this issue in the patch.

Best
-Paul


On Wed, Apr 24, 2019 at 6:08 PM Christopher Faulet 
wrote:

> Le 23/04/2019 à 11:10, Willy Tarreau a écrit :
> > Hi Paul,
> >
> > On Fri, Apr 19, 2019 at 06:45:22PM +0200, Paul Stephen Borile wrote:
> >> Hi Willy,
> >>
> >> fine for me, thanks for the adjustments and no problem backporting this
> to
> >> 1.9.
> >> I also confirm that the contact email address is working correctly.
> >
> > Fine thank you. I could finish the polishing (add USE_WURFL to the list
> > of known and reported build options in the makefile) and I've
> reintegrated
> > the code now.
> >
> > You probably don't see the value in having the dummy library, but for
> > developers it's invaluable. I have now updated my build script to build
> > with USE_WURFL=1 by default so that I'll now see if anything causes
> > warnings or errors there if we touch structures that are used by your
> > code.
> >
> > It would be really awesome if Device Atlas and 51Degrees could do the
> > same, as the build coverage becomes much better with very little effort
> > for everyone. David, Ben, if you read this, please have a look at
> > contrib/wurfl to get an idea of what is sufficient to have your code
> > always built by anyone. Patches welcome :-)
> >
> Hi Paul,
>
> I quickly reviewed the wurfl integration. I tested it with the dummy
> library. It really made my tests easier, many thanks.
>
> First, I have a segfault when I use the sample fetch "wurfl-get" at line
> 521. It happens when I try to retrieve an unknown data (I mean not
> listed in wurfl-information-list). Here is my config:
>
>  global
>  ...
>  wurfl-data-file /usr/share/wurfl/wurfl.zip
>  wurfl-information-list wurfl_id model_name
>  ...
>
>  frontend http
>  ...
>  http-request set-header X-WURFL-Properties
> %[wurfl-get(wurfl_id,is_tablet)] # is_tablet is not in the list
>  
>
> Then, at the beginning of the wurfl sample fetches, the channel validity
> must be checked calling the macro CHECK_HTTP_MESSAGE_FIRST(). Otherwise,
> some processing can be performed on empty buffers or uninitialized data.
>
> Finally, the function ha_wurfl_retrieve_header() is not HTX aware. Take
> a look at others HTTP sample fetches in src/http_fetch.c.
>
> Just a suggestion. It could be cool to call it from the dummy library,
> in wurfl_lookup(). This way, we will be able to test this part.
>
> Regards,
> --
> Christopher Faulet
>


-- 
Paul Stephen Borile
Director, WURFL InFuze Product Development - Italy
Scientiamobile Inc -  paul.bor...@scientiamobile.com +39 348 7108474
Milano Office : +39 02 620227260
skype: paulstephenborile
"Reducing complexity and size must be the goal in every step."  Niklaus
Wirth

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

On Wed, Apr 24, 2019 at 03:56:06PM +, Ben Shillito wrote:
> Thanks for the update. We will take a look and get a patch over to you.

Thank you Ben,
Willy

Re: [PATCH] wurfl device detection build fixes and dummy library

[Christopher Faulet]

Le 24/04/2019 à 17:56, Ben Shillito a écrit :

Hi Willy,

Thanks for the update. We will take a look and get a patch over to you.



Hi,

Ben, the function _51d_fetch() is not HTX aware. Take
a look at other HTTP sample fetches in src/http_fetch.c.

David, the same is true for the function da_haproxy_fetch().

Regards,
--
Christopher Faulet

Re: [PATCH] wurfl device detection build fixes and dummy library

[Christopher Faulet]

Le 23/04/2019 à 11:10, Willy Tarreau a écrit :

Hi Paul,

On Fri, Apr 19, 2019 at 06:45:22PM +0200, Paul Stephen Borile wrote:

Hi Willy,

fine for me, thanks for the adjustments and no problem backporting this to
1.9.
I also confirm that the contact email address is working correctly.


Fine thank you. I could finish the polishing (add USE_WURFL to the list
of known and reported build options in the makefile) and I've reintegrated
the code now.

You probably don't see the value in having the dummy library, but for
developers it's invaluable. I have now updated my build script to build
with USE_WURFL=1 by default so that I'll now see if anything causes
warnings or errors there if we touch structures that are used by your
code.

It would be really awesome if Device Atlas and 51Degrees could do the
same, as the build coverage becomes much better with very little effort
for everyone. David, Ben, if you read this, please have a look at
contrib/wurfl to get an idea of what is sufficient to have your code
always built by anyone. Patches welcome :-)


Hi Paul,

I quickly reviewed the wurfl integration. I tested it with the dummy 
library. It really made my tests easier, many thanks.


First, I have a segfault when I use the sample fetch "wurfl-get" at line 
521. It happens when I try to retrieve an unknown data (I mean not 
listed in wurfl-information-list). Here is my config:


global
...
wurfl-data-file /usr/share/wurfl/wurfl.zip
wurfl-information-list wurfl_id model_name
...

frontend http
...
http-request set-header X-WURFL-Properties 
%[wurfl-get(wurfl_id,is_tablet)] # is_tablet is not in the list



Then, at the beginning of the wurfl sample fetches, the channel validity 
must be checked calling the macro CHECK_HTTP_MESSAGE_FIRST(). Otherwise, 
some processing can be performed on empty buffers or uninitialized data.


Finally, the function ha_wurfl_retrieve_header() is not HTX aware. Take 
a look at others HTTP sample fetches in src/http_fetch.c.


Just a suggestion. It could be cool to call it from the dummy library, 
in wurfl_lookup(). This way, we will be able to test this part.


Regards,
--
Christopher Faulet

RE: [PATCH] wurfl device detection build fixes and dummy library

[Ben Shillito]

Hi Willy,

Thanks for the update. We will take a look and get a patch over to you.

Thanks,

Ben Shillito
Developer
O: +44 1183 287152
E: b...@51degrees.com
T: @51Degrees
-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu]
Sent: 23 April 2019 10:11
To: Paul Stephen Borile 
Cc: HAProxy 
Subject: Re: [PATCH] wurfl device detection build fixes and dummy library

Hi Paul,

On Fri, Apr 19, 2019 at 06:45:22PM +0200, Paul Stephen Borile wrote:
> Hi Willy,
>
> fine for me, thanks for the adjustments and no problem backporting
> this to 1.9.
> I also confirm that the contact email address is working correctly.

Fine thank you. I could finish the polishing (add USE_WURFL to the list of 
known and reported build options in the makefile) and I've reintegrated the 
code now.

You probably don't see the value in having the dummy library, but for 
developers it's invaluable. I have now updated my build script to build with 
USE_WURFL=1 by default so that I'll now see if anything causes warnings or 
errors there if we touch structures that are used by your code.

It would be really awesome if Device Atlas and 51Degrees could do the same, as 
the build coverage becomes much better with very little effort for everyone. 
David, Ben, if you read this, please have a look at contrib/wurfl to get an 
idea of what is sufficient to have your code always built by anyone. Patches 
welcome :-)

Thanks,
Willy

This email and any attachments are confidential and may also be privileged. If 
you are not the named recipient, please notify the sender immediately and do 
not disclose, use, store or copy the information contained herein. This is an 
email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 
118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees.

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

Hi Paul,

On Fri, Apr 19, 2019 at 06:45:22PM +0200, Paul Stephen Borile wrote:
> Hi Willy,
> 
> fine for me, thanks for the adjustments and no problem backporting this to
> 1.9.
> I also confirm that the contact email address is working correctly.

Fine thank you. I could finish the polishing (add USE_WURFL to the list
of known and reported build options in the makefile) and I've reintegrated
the code now.

You probably don't see the value in having the dummy library, but for
developers it's invaluable. I have now updated my build script to build
with USE_WURFL=1 by default so that I'll now see if anything causes
warnings or errors there if we touch structures that are used by your
code.

It would be really awesome if Device Atlas and 51Degrees could do the
same, as the build coverage becomes much better with very little effort
for everyone. David, Ben, if you read this, please have a look at
contrib/wurfl to get an idea of what is sufficient to have your code
always built by anyone. Patches welcome :-)

Thanks,
Willy

Re: [PATCH] wurfl device detection build fixes and dummy library

[Paul Stephen Borile]

Hi Willy,

fine for me, thanks for the adjustments and no problem backporting this to
1.9.
I also confirm that the contact email address is working correctly.

Thanks!
-Paul



On Fri, Apr 19, 2019 at 4:46 PM Willy Tarreau  wrote:

> Hi Paul,
>
> On Thu, Apr 18, 2019 at 02:46:17PM +0200, Paul Stephen Borile wrote:
> > please find attached to this email the 6 patches that cover various areas
> > of restyling of
> > the WURFL device detection feature for HAProxy. All patches can be back
> > ported to 1.9 if necessary.
> > Last patch is a dummy WURFL library that can be used to build/run haproxy
> > compiled with the USE_WURFL option to make easier checking for any build
> > problem in the future.
> > We'll try to do the same and make sure that the module does not break
> > builds again as happened in the past.
>
> So I gave a look to this patch set and had to perform a few adjustments
> to make it work but now it looks OK. I'm attaching the changes I made so
> that you can review them, they're all related to the dummy lib in order
> to 1) fix its build and 2) ease the testing without having to modify the
> build environment (since adding non-standard stuff into /usr/include or
> /usr/lib is a no-go on most development environments).
>
> I figured that it was much simpler to build a ".a" from the file so that
> it can naturally be loaded by the regular build process. I added the
> ability to report the libwurfl version in "haproxy -vv" and when the
> dummy lib is detected, it's explicitly mentioned "dummy library" there
> so that you don't have to deal with false positives when users report
> issues. I also added a little bit of doc explaining to haproxy devs
> how to build with wurfl. This way I think it could be added by default
> to any developer's build script so that it never breaks in the future.
>
> I'm attaching my changes. I'm fine with retrofitting them into your
> patches if they look OK to you. Please just let me know if you're OK to
> go with this (and if you're OK with me backporting this to 1.9 so that
> we can fix 1.9 once for all).
>
> Thanks!
> Willy
>
> PS: I've CCed the contact address in the maintainers file just to verify
> that there is no typo there, please confirm that it was properly
> received.
>
>

-- 
Paul Stephen Borile
Director, WURFL InFuze Product Development - Italy
Scientiamobile Inc -  paul.bor...@scientiamobile.com +39 348 7108474
Milano Office : +39 02 620227260
skype: paulstephenborile
"Reducing complexity and size must be the goal in every step."  Niklaus
Wirth

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

Sorry, with the patches this time.

Willy
>From 806aebcba5b7c5e3b780c63ff37a54b38a510e19 Mon Sep 17 00:00:00 2001
From: Willy Tarreau 
Date: Fri, 19 Apr 2019 16:12:08 +0200
Subject: [PATCH 07/13] WIP: wurfl: pass -fPIC when compiling

---
 contrib/wurfl/makefile | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/contrib/wurfl/makefile b/contrib/wurfl/makefile
index 56ee0f8cc..6822c94df 100644
--- a/contrib/wurfl/makefile
+++ b/contrib/wurfl/makefile
@@ -10,6 +10,9 @@ build: dummy-wurfl.o
rm -rf libwurfl.so ; ln -s libwurfl.so.0.1.10 libwurfl.so
rm -rf libwurfl.so.0 ; ln -s libwurfl.so.0.1.10 libwurfl.so.0
 
+dummy-wurfl.o: dummy-wurfl.c
+   $(CC) -fPIC -fvisibility=hidden -c dummy-wurfl.c
+
 install:
make build
mkdir -p /usr/include/wurfl; cp wurfl.h /usr/include/wurfl
-- 
2.20.1

>From 66d858a7ff657f4244e55d834628c5410b87c525 Mon Sep 17 00:00:00 2001
From: Willy Tarreau 
Date: Fri, 19 Apr 2019 16:12:40 +0200
Subject: [PATCH 08/13] WIP: wurfl: fix broken symlinks

.so.10 -> .so.100
---
 contrib/wurfl/makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/wurfl/makefile b/contrib/wurfl/makefile
index 6822c94df..0feb85e9f 100644
--- a/contrib/wurfl/makefile
+++ b/contrib/wurfl/makefile
@@ -7,8 +7,8 @@
 build: dummy-wurfl.o
$(CC) -fPIC -fvisibility=hidden -Wl,-Bsymbolic -Wl,--exclude-libs 
-Wl,ALL -shared -Wl,-soname \
-Wl,libwurfl.so.0 -o libwurfl.so.0.1.100 dummy-wurfl.o
-   rm -rf libwurfl.so ; ln -s libwurfl.so.0.1.10 libwurfl.so
-   rm -rf libwurfl.so.0 ; ln -s libwurfl.so.0.1.10 libwurfl.so.0
+   rm -rf libwurfl.so ; ln -s libwurfl.so.0.1.100 libwurfl.so
+   rm -rf libwurfl.so.0 ; ln -s libwurfl.so.0.1.100 libwurfl.so.0
 
 dummy-wurfl.o: dummy-wurfl.c
$(CC) -fPIC -fvisibility=hidden -c dummy-wurfl.c
-- 
2.20.1

>From 5aeb30470222877d7c705b731c6d8b46d3a171b9 Mon Sep 17 00:00:00 2001
From: Willy Tarreau 
Date: Fri, 19 Apr 2019 16:17:54 +0200
Subject: [PATCH 09/13] WIP: wurfl: address build issues by doing a static lib
 instead of a shared one

Here the symbols were not resolved, plus it's really not going to work to
have to install the dummy library on a developers' machine nor in an automatic
build environment. Instead let's just create libwurfl.a made from the single
.o file. The resulting binary will not depend on anything fancy this way.
---
 contrib/wurfl/makefile | 19 ---
 1 file changed, 4 insertions(+), 15 deletions(-)

diff --git a/contrib/wurfl/makefile b/contrib/wurfl/makefile
index 0feb85e9f..df08288ba 100644
--- a/contrib/wurfl/makefile
+++ b/contrib/wurfl/makefile
@@ -4,21 +4,10 @@
 #
 # install needs to be run as root
 
-build: dummy-wurfl.o
-   $(CC) -fPIC -fvisibility=hidden -Wl,-Bsymbolic -Wl,--exclude-libs 
-Wl,ALL -shared -Wl,-soname \
-   -Wl,libwurfl.so.0 -o libwurfl.so.0.1.100 dummy-wurfl.o
-   rm -rf libwurfl.so ; ln -s libwurfl.so.0.1.100 libwurfl.so
-   rm -rf libwurfl.so.0 ; ln -s libwurfl.so.0.1.100 libwurfl.so.0
+build: libwurfl.a
 
-dummy-wurfl.o: dummy-wurfl.c
-   $(CC) -fPIC -fvisibility=hidden -c dummy-wurfl.c
-
-install:
-   make build
-   mkdir -p /usr/include/wurfl; cp wurfl.h /usr/include/wurfl
-   cp libwurfl.so.0.1.100 /usr/lib
-   rm -rf /usr/lib/libwurfl.so ; ln -s /usr/lib/libwurfl.so.0.1.100 
/usr/lib/libwurfl.so
-   rm -rf /usr/lib/libwurfl.so.0 ; ln -s /usr/lib/libwurfl.so.0.1.100 
/usr/lib/libwurfl.so.0
+libwurfl.a: dummy-wurfl.o
+   ar rv $@ $<
 
 clean:
-   rm -rf *.so* *.o
+   rm -rf *.a *.o
-- 
2.20.1

>From f93f60bc04c847599bd57637f09df10bc9de54ff Mon Sep 17 00:00:00 2001
From: Willy Tarreau 
Date: Fri, 19 Apr 2019 16:28:53 +0200
Subject: [PATCH 10/13] WIP: wurfl: indicate in haproxy -vv the wurfl version
 in use and whether it's dummy or not

We have this output now :

$ ./haproxy  -vv |grep -i wurfl
Built with WURFL support (dummy library version 1.11.2.100)
---
 src/wurfl.c | 13 -
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/src/wurfl.c b/src/wurfl.c
index 118e58448..325cba649 100644
--- a/src/wurfl.c
+++ b/src/wurfl.c
@@ -702,6 +702,17 @@ static const char *ha_wurfl_retrieve_header(const char 
*header_name, const void
return ((ha_wurfl_header_t *)wh)->header_value;
 }
 
+static void ha_wurfl_register_build_options()
+{
+   const char *ver = wurfl_get_api_version();
+   char *ptr = NULL;
+
+   memprintf(, "Built with WURFL support (%sversion %s)",
+ strcmp(ver, "1.11.2.100") ? "" : "dummy library ",
+ ver);
+   hap_register_build_opts(ptr, 1);
+}
+
 REGISTER_POST_CHECK(ha_wurfl_init);
 REGISTER_POST_DEINIT(ha_wurfl_deinit);
-REGISTER_BUILD_OPTS("Built with WURFL support.");
+INITCALL0(STG_REGISTER, ha_wurfl_register_build_options);
-- 
2.20.1

>From 340ffa7d0b166ddde75dd864c28cc7513f0ab6ac Mon Sep 17 00:00:00 2001
From: Willy Tarreau 
Date: Fri, 19 Apr 2019 16:30:29 +0200
Subject: [PATCH 

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

Hi Paul,

On Thu, Apr 18, 2019 at 02:46:17PM +0200, Paul Stephen Borile wrote:
> please find attached to this email the 6 patches that cover various areas
> of restyling of
> the WURFL device detection feature for HAProxy. All patches can be back
> ported to 1.9 if necessary.
> Last patch is a dummy WURFL library that can be used to build/run haproxy
> compiled with the USE_WURFL option to make easier checking for any build
> problem in the future.
> We'll try to do the same and make sure that the module does not break
> builds again as happened in the past.

So I gave a look to this patch set and had to perform a few adjustments
to make it work but now it looks OK. I'm attaching the changes I made so
that you can review them, they're all related to the dummy lib in order
to 1) fix its build and 2) ease the testing without having to modify the
build environment (since adding non-standard stuff into /usr/include or
/usr/lib is a no-go on most development environments).

I figured that it was much simpler to build a ".a" from the file so that
it can naturally be loaded by the regular build process. I added the
ability to report the libwurfl version in "haproxy -vv" and when the
dummy lib is detected, it's explicitly mentioned "dummy library" there
so that you don't have to deal with false positives when users report
issues. I also added a little bit of doc explaining to haproxy devs
how to build with wurfl. This way I think it could be added by default
to any developer's build script so that it never breaks in the future.

I'm attaching my changes. I'm fine with retrofitting them into your
patches if they look OK to you. Please just let me know if you're OK to
go with this (and if you're OK with me backporting this to 1.9 so that
we can fix 1.9 once for all).

Thanks!
Willy

PS: I've CCed the contact address in the maintainers file just to verify
that there is no typo there, please confirm that it was properly
received.

Re: [PATCH] wurfl device detection build fixes and dummy library

[Willy Tarreau]

Hi Paul,

On Thu, Apr 18, 2019 at 02:46:17PM +0200, Paul Stephen Borile wrote:
> Hi All, Willy,
> 
> please find attached to this email the 6 patches that cover various areas
> of restyling of
> the WURFL device detection feature for HAProxy. All patches can be back
> ported to 1.9 if necessary.
> Last patch is a dummy WURFL library that can be used to build/run haproxy
> compiled with the USE_WURFL option to make easier checking for any build
> problem in the future.
> We'll try to do the same and make sure that the module does not break
> builds again as happened in the past.

Many thanks for taking care of this. I'll have a look at it ASAP.

Thanks,
Willy