Re: Does anyone *really* use 51d or WURFL ?
> > One could argue that 1) not building, 2) not working, and 3) > not being maintained doesn't exactly qualify as "stable". So maybe in the > end I'll do it there as well. And I'm sure it will not affect distros! > But I'm open to opinions on the subject. > > This seems to go against #1 quality of HAProxy: reliability... So you have my +1 :) Baptiste
Re: Does anyone *really* use 51d or WURFL ?
Hi. Am 05.03.2019 um 13:47 schrieb Willy Tarreau: > Hi all, > > back to this old thread : > > On Mon, Jan 21, 2019 at 03:36:22PM +0100, Willy Tarreau wrote: >> I don't know if wurfl builds at all by the way since the last update to >> the module is its introduction more than 2 years ago. > > So now at least I've got the response. This code doesn't build and was > broken no less than 9 months ago without anyone ever noticing. And it > never cared to try to be thread-compatible over the last 28 months. > This is not a feature but a useless heavy weight we're carrying behind > us for no valid reason. Given that it hasn't ever been maintained since > its introduction and obviously doesn't even have a user, I've removed > it. I'd be tempted to backport the cleanup patch to 1.9 since it doesn't > build there either, but I don't like to remove features in stable > branches. One could argue that 1) not building, 2) not working, and 3) > not being maintained doesn't exactly qualify as "stable". So maybe in the > end I'll do it there as well. And I'm sure it will not affect distros! > But I'm open to opinions on the subject. >From my point of view if even the contributor does not care that this feature works well, why should we (the community) care? The 51d have answered quite fast with a fix. >From haproxy point of view I would add a dedicated note and a Warning that this feature does not work in the current version. If anyone want to fix this step forward and fix it. I would remove it in 2.x. Jm2c > Regards, > Willy Regards Aleks
Re: Does anyone *really* use 51d or WURFL ?
Hi all, back to this old thread : On Mon, Jan 21, 2019 at 03:36:22PM +0100, Willy Tarreau wrote: > I don't know if wurfl builds at all by the way since the last update to > the module is its introduction more than 2 years ago. So now at least I've got the response. This code doesn't build and was broken no less than 9 months ago without anyone ever noticing. And it never cared to try to be thread-compatible over the last 28 months. This is not a feature but a useless heavy weight we're carrying behind us for no valid reason. Given that it hasn't ever been maintained since its introduction and obviously doesn't even have a user, I've removed it. I'd be tempted to backport the cleanup patch to 1.9 since it doesn't build there either, but I don't like to remove features in stable branches. One could argue that 1) not building, 2) not working, and 3) not being maintained doesn't exactly qualify as "stable". So maybe in the end I'll do it there as well. And I'm sure it will not affect distros! But I'm open to opinions on the subject. Regards, Willy
Re: Does anyone *really* use 51d or WURFL ?
Hi Ben, On Tue, Feb 05, 2019 at 01:37:59PM +, Ben Shillito wrote: > Hi Willy, > > I have attached two patches. > > One is the threading change which maps the threading flag in 51Degrees to the > one in HAProxy. There are also some changes in the 51d.c module code to make > everything thread safe. > > The other is a minor bug in the multiple header matching when using the Hash > Trie API. Thanks and sorry for the delay, I thought I already picked them. Now merged. Thanks, Willy
RE: Does anyone *really* use 51d or WURFL ?
Hi Willy, I have attached two patches. One is the threading change which maps the threading flag in 51Degrees to the one in HAProxy. There are also some changes in the 51d.c module code to make everything thread safe. The other is a minor bug in the multiple header matching when using the Hash Trie API. Thanks, Ben Shillito Developer O: +44 1183 287152 E: b...@51degrees.com T: @51Degrees -Original Message- From: Willy Tarreau [mailto:w...@1wt.eu] Sent: 21 January 2019 16:48 To: Ben Shillito Cc: haproxy@formilux.org Subject: Re: Does anyone *really* use 51d or WURFL ? On Mon, Jan 21, 2019 at 04:37:32PM +, Ben Shillito wrote: > Hi Willy, > > Ah yes, thanks, I missed the S first time reading it. > > There are actually a couple of things I'd like to check over a bit > more thoroughly like the caching used in 51d.c, so it will probably be > more like tomorrow. No problem. We'll probably issue another 1.9 on wednesday. If you can have something by then it could be backported into the next 1.9, otherwise it can still wait a bit. It's been waiting for more than one year, we're not in a hurry anymore :-) Thanks, Willy This email and any attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender immediately and do not disclose, use, store or copy the information contained herein. This is an email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees. 0001-BUG-51d-In-Hash-Trie-multi-header-matching-was-affec.patch Description: 0001-BUG-51d-In-Hash-Trie-multi-header-matching-was-affec.patch 0002-FEAT-51d-Enabled-multi-threaded-operation-in-the-51D.patch Description: 0002-FEAT-51d-Enabled-multi-threaded-operation-in-the-51D.patch
Re: Does anyone *really* use 51d or WURFL ?
On Mon, Jan 21, 2019 at 04:37:32PM +, Ben Shillito wrote: > Hi Willy, > > Ah yes, thanks, I missed the S first time reading it. > > There are actually a couple of things I'd like to check over a bit more > thoroughly like the caching used in 51d.c, so it will probably be more like > tomorrow. No problem. We'll probably issue another 1.9 on wednesday. If you can have something by then it could be backported into the next 1.9, otherwise it can still wait a bit. It's been waiting for more than one year, we're not in a hurry anymore :-) Thanks, Willy
RE: Does anyone *really* use 51d or WURFL ?
Hi Willy, Ah yes, thanks, I missed the S first time reading it. There are actually a couple of things I'd like to check over a bit more thoroughly like the caching used in 51d.c, so it will probably be more like tomorrow. Thanks, Ben Shillito Developer O: +44 1183 287152 E: b...@51degrees.com T: @51Degrees -Original Message- From: Willy Tarreau [mailto:w...@1wt.eu] Sent: 21 January 2019 16:06 To: Ben Shillito Cc: haproxy@formilux.org Subject: Re: Does anyone *really* use 51d or WURFL ? On Mon, Jan 21, 2019 at 04:00:13PM +, Ben Shillito wrote: > Hi Willy, > > I agree, setting the flag from the HAProxy USE_THREADS is probably the > neatest solution. Yep. Be careful, it's "USE_THREAD" (without trailing S). > I will get a patch over to you later on today. Fine, no emergency though. I prefer that you take the time to verify that it works before submitting :-) Thanks, Willy This email and any attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender immediately and do not disclose, use, store or copy the information contained herein. This is an email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees.
Re: Does anyone *really* use 51d or WURFL ?
Hi Patrick, On Mon, Jan 21, 2019 at 10:54:17AM -0500, Patrick Hemmer wrote: > We do use 51Degrees at my place of employment. However a couple of > caveats in that statement. Great, thank you for the feedback! > One is that we're still running on 1.7. No problem. > We'll > likely be upgrading to 1.9 soon, but still a couple months out. That's definitely a reasonable approach. > The other caveat is that we run with threading disabled. Until the > statement on 'nbthread' that "THREADS SUPPORT IN HAPROXY IS HIGHLY > EXPERIMENTAL" goes away, we'll be leaving it off. Ah good catch, that's always the problem when placing statements like this in the doc at a place that has no chance for being revisited! I'll remove it for -dev and 1.9. That was totally true at the time when 1.8 was released however, but now thread-specific issues are much less common than other ones. I consider that 1.8 is safe now regarding threads, however they're suboptimal and the scalability isn't that great above ~4. I should possibly revisit the statement there as well to reflect this. Thanks! willy
Re: Does anyone *really* use 51d or WURFL ?
On Mon, Jan 21, 2019 at 04:00:13PM +, Ben Shillito wrote: > Hi Willy, > > I agree, setting the flag from the HAProxy USE_THREADS is probably the > neatest solution. Yep. Be careful, it's "USE_THREAD" (without trailing S). > I will get a patch over to you later on today. Fine, no emergency though. I prefer that you take the time to verify that it works before submitting :-) Thanks, Willy
RE: Does anyone *really* use 51d or WURFL ?
Hi Willy, I agree, setting the flag from the HAProxy USE_THREADS is probably the neatest solution. I will get a patch over to you later on today. Thanks, Ben Shillito Developer O: +44 1183 287152 E: b...@51degrees.com T: @51Degrees -Original Message- From: Willy Tarreau [mailto:w...@1wt.eu] Sent: 21 January 2019 15:44 To: Ben Shillito Cc: haproxy@formilux.org Subject: Re: Does anyone *really* use 51d or WURFL ? Hi Ben, First, thanks for your quick response. On Mon, Jan 21, 2019 at 03:05:08PM +, Ben Shillito wrote: > Hi Willy, > > I'd like to point out that the 51Degrees API does in fact support > multi-threaded operation by default. The HAProxy makefile however, > explicitly uses the FIFTYONEDEGREES_NO_THREADING compile option to > disable this when building > (https://github.com/haproxy/haproxy/blob/master/Makefile#L740). OK! > Multi-threaded operation in the API is tested under maximum load > before every release as can be seen here: > https://github.com/51Degrees/Device-Detection/blob/master/VisualStudio > /UnitTests/Performance/Base.cs#L89 > > Mutli-threaded operation is also stress tested in conjunction with > many threads detecting User-Agents, and another reloading the data > file repeatedly as seen here: > https://github.com/51Degrees/Device-Detection/blob/master/examples/Rel > oadFromFile.c Oh, rest assured that I have no doubt about the API's support itself, what I mean is that 51d.c in haproxy is not thread-compatible right now (at least due to the makefile entry above and to the test in 51d.c) while haproxy is built with threading on by default, and I find it suspicious that nobody noticed after more than one year. But it's very possible that for now nobody is interested in deploying it with threads. At the moment I'm concerned by the fact that in the whole project there remains exactly two files which are incompatible with thread support and as you can imagine it's always a pain to have to deal with exceptions like this. > If multi-threaded support is an issue, then it is a trivial change to > the makefile to align the HAProxy build to the 51Degrees default, and > give the option to disable threading for those who require that. Excellent! What I can suggest then is to set this define only when USE_THREAD is not set. This way you can make sure that 51d.c and the rest of haproxy are always aligned regarding thread support. Feel free to send a patch, I'll happily merge it and even backport it so that we don't have to worry anymore about the compatbility between threads and various options. Thanks! Willy This email and any attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender immediately and do not disclose, use, store or copy the information contained herein. This is an email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees.
Re: Does anyone *really* use 51d or WURFL ?
On 2019/1/21 09:36, Willy Tarreau wrote: > Hi all, > > recently it was figured that the buffer API changes caused some breakage > to da.c and 51d.c (both fixed since), I don't know if wurfl builds at all > by the way since the last update to the module is its introduction more > than 2 years ago. But more importantly I'm realizing that neither 51d nor > wurfl will start with threads enabled. This has been the case for about > 15 months now, while we've had threads enabled on all relevant operating > systems. > > Thus it leads me to the (possibly wrong) conclusion that absolutely > nobody ever uses these options. Am I wrong ? I'm asking because each > time we perform some API changes it's always a pain to go through these > which don't build out of the box without external dependencies, and I > suspect we're simply wasting our time and we should get rid of them > if nobody uses them (or at the very least move them to contrib/). > > But if anyone is using them and disables threads for this, then it's > fine, but in this case it would be nice if these two would check the > thread safety of their code so that the thread restriction can be > removed for the benefit of their users. > > Please advise. > > Thanks, > Willy > We do use 51Degrees at my place of employment. However a couple of caveats in that statement. One is that we're still running on 1.7. We'll likely be upgrading to 1.9 soon, but still a couple months out. The other caveat is that we run with threading disabled. Until the statement on 'nbthread' that "THREADS SUPPORT IN HAPROXY IS HIGHLY EXPERIMENTAL" goes away, we'll be leaving it off. -Patrick
Re: Does anyone *really* use 51d or WURFL ?
Hi Ben, First, thanks for your quick response. On Mon, Jan 21, 2019 at 03:05:08PM +, Ben Shillito wrote: > Hi Willy, > > I'd like to point out that the 51Degrees API does in fact support > multi-threaded operation by default. The HAProxy makefile however, explicitly > uses the FIFTYONEDEGREES_NO_THREADING compile option to disable this when > building (https://github.com/haproxy/haproxy/blob/master/Makefile#L740). OK! > Multi-threaded operation in the API is tested under maximum load before every > release as can be seen here: > https://github.com/51Degrees/Device-Detection/blob/master/VisualStudio/UnitTests/Performance/Base.cs#L89 > > Mutli-threaded operation is also stress tested in conjunction with many > threads detecting User-Agents, and another reloading the data file repeatedly > as seen here: > https://github.com/51Degrees/Device-Detection/blob/master/examples/ReloadFromFile.c Oh, rest assured that I have no doubt about the API's support itself, what I mean is that 51d.c in haproxy is not thread-compatible right now (at least due to the makefile entry above and to the test in 51d.c) while haproxy is built with threading on by default, and I find it suspicious that nobody noticed after more than one year. But it's very possible that for now nobody is interested in deploying it with threads. At the moment I'm concerned by the fact that in the whole project there remains exactly two files which are incompatible with thread support and as you can imagine it's always a pain to have to deal with exceptions like this. > If multi-threaded support is an issue, then it is a trivial change to the > makefile to align the HAProxy build to the 51Degrees default, and give the > option to disable threading for those who require that. Excellent! What I can suggest then is to set this define only when USE_THREAD is not set. This way you can make sure that 51d.c and the rest of haproxy are always aligned regarding thread support. Feel free to send a patch, I'll happily merge it and even backport it so that we don't have to worry anymore about the compatbility between threads and various options. Thanks! Willy
RE: Does anyone *really* use 51d or WURFL ?
Hi Willy, I'd like to point out that the 51Degrees API does in fact support multi-threaded operation by default. The HAProxy makefile however, explicitly uses the FIFTYONEDEGREES_NO_THREADING compile option to disable this when building (https://github.com/haproxy/haproxy/blob/master/Makefile#L740). Multi-threaded operation in the API is tested under maximum load before every release as can be seen here: https://github.com/51Degrees/Device-Detection/blob/master/VisualStudio/UnitTests/Performance/Base.cs#L89 Mutli-threaded operation is also stress tested in conjunction with many threads detecting User-Agents, and another reloading the data file repeatedly as seen here: https://github.com/51Degrees/Device-Detection/blob/master/examples/ReloadFromFile.c If multi-threaded support is an issue, then it is a trivial change to the makefile to align the HAProxy build to the 51Degrees default, and give the option to disable threading for those who require that. Regards, Ben Shillito Developer O: +44 1183 287152 E: b...@51degrees.com T: @51Degrees -Original Message- From: Willy Tarreau [mailto:w...@1wt.eu] Sent: 21 January 2019 14:36 To: haproxy@formilux.org Subject: Does anyone *really* use 51d or WURFL ? Hi all, recently it was figured that the buffer API changes caused some breakage to da.c and 51d.c (both fixed since), I don't know if wurfl builds at all by the way since the last update to the module is its introduction more than 2 years ago. But more importantly I'm realizing that neither 51d nor wurfl will start with threads enabled. This has been the case for about 15 months now, while we've had threads enabled on all relevant operating systems. Thus it leads me to the (possibly wrong) conclusion that absolutely nobody ever uses these options. Am I wrong ? I'm asking because each time we perform some API changes it's always a pain to go through these which don't build out of the box without external dependencies, and I suspect we're simply wasting our time and we should get rid of them if nobody uses them (or at the very least move them to contrib/). But if anyone is using them and disables threads for this, then it's fine, but in this case it would be nice if these two would check the thread safety of their code so that the thread restriction can be removed for the benefit of their users. Please advise. Thanks, Willy This email and any attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender immediately and do not disclose, use, store or copy the information contained herein. This is an email from 51Degrees.mobi Limited, 5 Charlotte Close, Reading. RG47BY. T: +44 118 328 7152; E: i...@51degrees.com; 51Degrees.mobi Limited t/as 51Degrees.
