Re: ARM(64) builds
вс, 19 янв. 2020 г. в 13:13, Martin Grigorov : > Hi, > > On Sat, Jan 18, 2020, 22:10 Илья Шипицин wrote: > >> tests on ARM64 randomly fail >> https://travis-ci.com/chipitsine/haproxy/jobs/277236120 >> >> (after restart there's a good chance to success) >> > > I have the same observation on TravisCI. I think the reason is that their > arm64 instances are less powerful than the amd64 ones. > At https://docs.travis-ci.com/user/multi-cpu-architectures it is said: > While available to all Open Source repositories, the concurrency available > for multiple CPU arch-based jobs is limited during the alpha period. > Not sure how much limited it is though. > I'd like to investigate that on dedicated ARM64 server (which I currently does not have). I applied for one at Linode, no answer yet. also, it might depend on number of CPU (race condition likely reproduce on many CPU than on single CPU server). also, arm64 seems not very happy about our linker options /usr/bin/ld: src/ev_poll.o(.debug_info+0x78): R_AARCH64_ABS64 used with TLS symbol poll_events > > Today this test failed once on my VM: > > #top TEST reg-tests/seamless-reload/abns_socket.vtc FAILED (1.111) > exit=2 > 1 tests failed, 0 tests skipped, 34 tests passed > ## Gathering results ## > ## Test case: reg-tests/seamless-reload/abns_socket.vtc ## > ## test results in: > "/tmp/haregtests-2020-01-19_08-06-39.45Hchw/vtc.8496.328d7f95" > c1 HTTP rx failed (fd:6 read: Connection reset by peer) > Makefile:964: recipe for target 'reg-tests' failed > make: *** [reg-tests] Error 1 > > but the next 4 runs were successul. > > >> сб, 18 янв. 2020 г. в 09:52, Martin Grigorov : >> >>> >>> >>> On Fri, Jan 17, 2020 at 11:17 PM Martin Grigorov < >>> martin.grigo...@gmail.com> wrote: >>> On Fri, Jan 17, 2020, 23:12 William Lallemand wrote: > On Fri, Jan 17, 2020 at 08:50:27PM +0200, Martin Grigorov wrote: > > Testing with haproxy version: 2.2-dev0-70c5b0-123 > > This binary was built with code from 1 week ago, it's normal that the > test does > not work since the fix was made this week. > I'm using the same steps to build HAProxy as from .travis-ci.yml I guess I have to add "make clean" in the beginning. I'll try it tomorrow! Thanks! >>> >>> That was it! >>> Everything is back to normal now! >>> Thank you, William! >>> >>> > -- > William Lallemand >
Re: ARM(64) builds
Hi, On Sat, Jan 18, 2020, 22:10 Илья Шипицин wrote: > tests on ARM64 randomly fail > https://travis-ci.com/chipitsine/haproxy/jobs/277236120 > > (after restart there's a good chance to success) > I have the same observation on TravisCI. I think the reason is that their arm64 instances are less powerful than the amd64 ones. At https://docs.travis-ci.com/user/multi-cpu-architectures it is said: While available to all Open Source repositories, the concurrency available for multiple CPU arch-based jobs is limited during the alpha period. Not sure how much limited it is though. Today this test failed once on my VM: #top TEST reg-tests/seamless-reload/abns_socket.vtc FAILED (1.111) exit=2 1 tests failed, 0 tests skipped, 34 tests passed ## Gathering results ## ## Test case: reg-tests/seamless-reload/abns_socket.vtc ## ## test results in: "/tmp/haregtests-2020-01-19_08-06-39.45Hchw/vtc.8496.328d7f95" c1 HTTP rx failed (fd:6 read: Connection reset by peer) Makefile:964: recipe for target 'reg-tests' failed make: *** [reg-tests] Error 1 but the next 4 runs were successul. > сб, 18 янв. 2020 г. в 09:52, Martin Grigorov : > >> >> >> On Fri, Jan 17, 2020 at 11:17 PM Martin Grigorov < >> martin.grigo...@gmail.com> wrote: >> >>> >>> >>> On Fri, Jan 17, 2020, 23:12 William Lallemand >>> wrote: >>> On Fri, Jan 17, 2020 at 08:50:27PM +0200, Martin Grigorov wrote: > Testing with haproxy version: 2.2-dev0-70c5b0-123 This binary was built with code from 1 week ago, it's normal that the test does not work since the fix was made this week. >>> >>> I'm using the same steps to build HAProxy as from .travis-ci.yml >>> I guess I have to add "make clean" in the beginning. >>> I'll try it tomorrow! Thanks! >>> >> >> That was it! >> Everything is back to normal now! >> Thank you, William! >> >> >>> >>> -- William Lallemand >>>
Re: ARM(64) builds
tests on ARM64 randomly fail https://travis-ci.com/chipitsine/haproxy/jobs/277236120 (after restart there's a good chance to success) сб, 18 янв. 2020 г. в 09:52, Martin Grigorov : > > > On Fri, Jan 17, 2020 at 11:17 PM Martin Grigorov < > martin.grigo...@gmail.com> wrote: > >> >> >> On Fri, Jan 17, 2020, 23:12 William Lallemand >> wrote: >> >>> On Fri, Jan 17, 2020 at 08:50:27PM +0200, Martin Grigorov wrote: >>> > Testing with haproxy version: 2.2-dev0-70c5b0-123 >>> >>> This binary was built with code from 1 week ago, it's normal that the >>> test does >>> not work since the fix was made this week. >>> >> >> I'm using the same steps to build HAProxy as from .travis-ci.yml >> I guess I have to add "make clean" in the beginning. >> I'll try it tomorrow! Thanks! >> > > That was it! > Everything is back to normal now! > Thank you, William! > > >> >> >>> -- >>> William Lallemand >>> >>
Re: ARM(64) builds
On Fri, Jan 17, 2020 at 11:17 PM Martin Grigorov wrote: > > > On Fri, Jan 17, 2020, 23:12 William Lallemand > wrote: > >> On Fri, Jan 17, 2020 at 08:50:27PM +0200, Martin Grigorov wrote: >> > Testing with haproxy version: 2.2-dev0-70c5b0-123 >> >> This binary was built with code from 1 week ago, it's normal that the >> test does >> not work since the fix was made this week. >> > > I'm using the same steps to build HAProxy as from .travis-ci.yml > I guess I have to add "make clean" in the beginning. > I'll try it tomorrow! Thanks! > That was it! Everything is back to normal now! Thank you, William! > > >> -- >> William Lallemand >> >
Re: ARM(64) builds
On Fri, Jan 17, 2020, 23:12 William Lallemand wrote: > On Fri, Jan 17, 2020 at 08:50:27PM +0200, Martin Grigorov wrote: > > Testing with haproxy version: 2.2-dev0-70c5b0-123 > > This binary was built with code from 1 week ago, it's normal that the test > does > not work since the fix was made this week. > I'm using the same steps to build HAProxy as from .travis-ci.yml I guess I have to add "make clean" in the beginning. I'll try it tomorrow! Thanks! > -- > William Lallemand >
Re: ARM(64) builds
On Fri, Jan 17, 2020 at 08:50:27PM +0200, Martin Grigorov wrote: > Testing with haproxy version: 2.2-dev0-70c5b0-123 This binary was built with code from 1 week ago, it's normal that the test does not work since the fix was made this week. -- William Lallemand
Re: ARM(64) builds
On Fri, Jan 17, 2020 at 08:50:27PM +0200, Martin Grigorov wrote: > Hi William, > > Please find attached the output. > > this two lines look bad in it: > > *** h1 debug|[ALERT] 016/184150 (7188) : parsing [cur--1:0] : proxy > 'MASTER', another server named 'cur--1' was already defined at line 0, > please use distinct names. > *** h1 debug|[ALERT] 016/184150 (7188) : Fatal errors found in > configuration. > Are you sure your binary is up to date? because the fix for this exact problem is the commit just before the one that created the regtest: https://github.com/haproxy/haproxy/commit/a31b09e982a76cdf8761edb25d1569cb76a8ff37 -- William Lallemand
Re: ARM(64) builds
Hi Илья, On Fri, Jan 17, 2020 at 5:43 PM Илья Шипицин wrote: > > > пт, 17 янв. 2020 г. в 19:33, Martin Grigorov : > >> >> >> On Fri, Jan 17, 2020 at 4:13 PM Martin Grigorov < >> martin.grigo...@gmail.com> wrote: >> >>> Hi all, >>> >>> Today's build consistently fails on my ARM64 VM: >>> >>> ## Starting vtest ## >>> Testing with haproxy version: 2.2-dev0-70c5b0-123 >>> #top TEST reg-tests/mcli/mcli_start_progs.vtc FAILED (3.004) exit=2 >>> 1 tests failed, 0 tests skipped, 34 tests passed >>> ## Gathering results ## >>> ## Test case: reg-tests/mcli/mcli_start_progs.vtc ## >>> ## test results in: >>> "/tmp/haregtests-2020-01-17_14-01-45.SGkYcJ/vtc.12807.6adfff44" >>> h1 haproxy h1 PID file check failed: >>> h1 Bad exit status: 0x0100 exit 0x1 signal 0 core 0 >>> Makefile:964: recipe for target 'reg-tests' failed >>> make: *** [reg-tests] Error 1 >>> >> >> git bisect blaims this commit: >> >> 25b569302167e71b32e569a2366027e8e320e80a is the first bad commit >> commit 25b569302167e71b32e569a2366027e8e320e80a >> Author: William Lallemand >> Date: Tue Jan 14 15:38:43 2020 +0100 >> >> REGTEST: mcli/mcli_start_progs: start 2 programs >> >> This regtest tests the issue #446 by starting 2 programs and checking >> if >> they exist in the "show proc" of the master CLI. >> >> Should be backported as far as 2.0. >> >> >> https://travis-ci.com/haproxy/haproxy is green >> https://cirrus-ci.com/github/haproxy/haproxy is green >> and what is even more interesting is that >> https://travis-ci.org/martin-g/haproxy/builds (my fork with enabled >> ARM64 testing on TravisCI) also just passed (after few failures due to >> timing issues (I guess) >> > > > timing out might happen because of > https://github.com/haproxy/haproxy/commit/ac8147446c7a3d1aa607042bc782095b03bc8dc4 > your fork is 16 commits behind current master. try to rebase to master > Thanks! I've updated HAProxy on my ARM64 VM but didn't update my fork at GitHub so it was behind. I've just did it and the build again passed successfully: https://travis-ci.org/martin-g/haproxy/builds/638568217 > > >> >> >>> Regards, >>> Martin >>> >>> On Fri, Jan 17, 2020 at 9:22 AM Илья Шипицин >>> wrote: >>> привет! пт, 17 янв. 2020 г. в 11:42, Martin Grigorov >>> >: > Привет Илья, > > On Thu, Jan 16, 2020 at 10:37 AM Илья Шипицин > wrote: > >> >> >> чт, 16 янв. 2020 г. в 13:26, Martin Grigorov < >> martin.grigo...@gmail.com>: >> >>> Hi Илья, >>> >>> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин >>> wrote: >>> Hello, Martin! btw, just curious, how is Apache Foundation (or you personally) interested in all that ? please do not blame me, I really like to know. >>> >> ok, so you work in some company that is interested in haproxy on >> ARM64. >> you do not want to tell it's name, at least is it legal ? is it >> related to some government ? >> if "no" and "no", I guess most people won't ask any more questions :) >> > > It is legal and I do not work for a government of any country! > > >> >> personally, I do not work at Haproxy Inc, I use haproxy, sometimes I >> contribute to it. >> Please do not consider me as an "official representative". >> >> >> I'm interested in testing haproxy on ARM64, I planned to do so. I can >> priorierize it according to your interest to it. >> and yes, I can accept hardware donation (even considering I'm not >> part of Haproxy Inc). >> >> also, from my point of view, what would be really useful in your case >> is testing not just official reg-tests, but also >> your configs. reg-tests cover only partially. If you enable clang >> asan in your own workload there's a chance to catch >> something interesting (or, to make sure your own workload is ok). I >> can help with that as well. >> > > Thanks for the offer! > I've discussed it with my managers. > Our offer is to donate a VM that could be used as an official CI agent > for the HAProxy project long term. > I'd split this into short term and long term approach. if you need to start any time soon, I'd focus on your own workload testing first. I would build stand which emulates your production workload, compile haproxy using clang address sanitizer and give it a try (functional testing, load testing, ...) I can help with that. As for long term solution, currently haproxy simply cannot attach any dedicated build agent to their CI. travis-ci does not allow attaching dedicated agents. And haproxy team is very conservative in adding new CI servers. I think, I will add arm64 (most probably openssl-1.1.1 only for now)
Re: ARM(64) builds
Hi William,
On Fri, Jan 17, 2020 at 4:46 PM William Lallemand
wrote:
> On Fri, Jan 17, 2020 at 04:33:22PM +0200, Martin Grigorov wrote:
> >
> > git bisect blaims this commit:
> >
> > 25b569302167e71b32e569a2366027e8e320e80a is the first bad commit
> > commit 25b569302167e71b32e569a2366027e8e320e80a
> > Author: William Lallemand
> > Date: Tue Jan 14 15:38:43 2020 +0100
> >
> > REGTEST: mcli/mcli_start_progs: start 2 programs
>
> Well that's the commit which introduces the vtc file so that's normal.
>
> >
> > https://travis-ci.com/haproxy/haproxy is green
> > https://cirrus-ci.com/github/haproxy/haproxy is green
> > and what is even more interesting is that
> > https://travis-ci.org/martin-g/haproxy/builds (my fork with enabled
> ARM64
> > testing on TravisCI) also just passed (after few failures due to timing
> > issues (I guess)
>
> I don't see anything regarding mcli_start_progs.vtc in this links, can you
> provide the output of
> `make reg-tests -- --debug reg-tests/mcli/mcli_start_progs.vtc` ?
>
Please find attached the output.
this two lines look bad in it:
*** h1 debug|[ALERT] 016/184150 (7188) : parsing [cur--1:0] : proxy
'MASTER', another server named 'cur--1' was already defined at line 0,
please use distinct names.
*** h1 debug|[ALERT] 016/184150 (7188) : Fatal errors found in
configuration.
$ grep -rnH 'cur'
/tmp/haregtests-2020-01-17_18-41-50.A6zVLb/vtc.7182.700b0bcd/
/tmp/haregtests-2020-01-17_18-41-50.A6zVLb/vtc.7182.700b0bcd/LOG:71:*** h1
debug|[ALERT] 016/184150 (7188) : parsing [cur--1:0] : proxy 'MASTER',
another server named 'cur--1' was already defined at line 0, please use
distinct names.
│ File:
/tmp/haregtests-2020-01-17_18-41-50.A6zVLb/vtc.7182.700b0bcd/h1/cfg
───┼─
1 │ global
2 │ stats socket
"/tmp/haregtests-2020-01-17_18-41-50.A6zVLb/vtc.7182.700b0bcd/h1/stats.sock"
level admin mode 600
3 │ stats socket "fd@${cli}" level admin
4 │
5 │ global
6 │ nbproc 1
7 │ defaults
8 │ mode http
9 │ option http-use-htx
10 │ timeout connect 1s
11 │ timeout client 1s
12 │ timeout server 1s
13 │
14 │ frontend myfrontend
15 │ bind "fd@${my_fe}"
16 │ default_backend test
17 │
18 │ backend test
19 │ server www1 127.0.0.1:39247
20 │
21 │ program foo
22 │ command sleep 10
23 │
24 │ program bar
25 │ command sleep 10
Please let me know if you need more information!
Thanks
>
> --
> William Lallemand
>
env VTEST_PROGRAM=../vtest/vtest make reg-tests -- --debug
reg-tests/mcli/mcli_start_progs.vtc
## Preparing to run tests ##
Testing with haproxy version: 2.2-dev0-70c5b0-123
Target : linux-glibc
Options : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER -PCRE -PCRE_JIT -PCRE2
-PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED -REGPARM -STATIC_PCRE
-STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -VSYSCALL
+GETADDRINFO -OPENSSL -LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY
+TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER
+PRCTL +THREAD_DUMP -EVPORTS
## Gathering tests to run ##
Add test: reg-tests/mcli/mcli_start_progs.vtc
## Starting vtest ##
Testing with haproxy version: 2.2-dev0-70c5b0-123
dT 0.000
*top TEST reg-tests/mcli/mcli_start_progs.vtc starting
top extmacro def pwd=/home/ubuntu/git/haproxy/haproxy
top extmacro def no-htx=
top extmacro def localhost=127.0.0.1
top extmacro def bad_backend=127.0.0.1 45249
top extmacro def bad_ip=192.0.2.255
top macro def testdir=/home/ubuntu/git/haproxy/haproxy/reg-tests/mcli
top macro def
tmpdir=/tmp/haregtests-2020-01-17_18-41-50.A6zVLb/vtc.7182.700b0bcd
** top === varnishtest "Try to start a master CLI with 2 programs"
*top VTEST Try to start a master CLI with 2 programs
** top === feature ignore_unknown_macro
** top === server s1 {
** s1 Starting server
s1 macro def s1_addr=127.0.0.1
s1 macro def s1_port=39247
s1 macro def s1_sock=127.0.0.1 39247
*s1 Listen on 127.0.0.1 39247
** top === haproxy h1 -W -S -conf {
** s1 Started on 127.0.0.1 39247 (1 iterations)
h1 macro def h1_closed_sock=127.0.0.1 33279
h1 macro def h1_closed_addr=127.0.0.1
h1 macro def h1_closed_port=33279
dT 0.003
h1 macro def h1_cli_sock=127.0.0.1 45381
h1 macro def h1_cli_addr=127.0.0.1
h1 macro def h1_cli_port=45381
h1 setenv(cli, 6)
h1 macro def
Re: ARM(64) builds
пт, 17 янв. 2020 г. в 19:33, Martin Grigorov : > > > On Fri, Jan 17, 2020 at 4:13 PM Martin Grigorov > wrote: > >> Hi all, >> >> Today's build consistently fails on my ARM64 VM: >> >> ## Starting vtest ## >> Testing with haproxy version: 2.2-dev0-70c5b0-123 >> #top TEST reg-tests/mcli/mcli_start_progs.vtc FAILED (3.004) exit=2 >> 1 tests failed, 0 tests skipped, 34 tests passed >> ## Gathering results ## >> ## Test case: reg-tests/mcli/mcli_start_progs.vtc ## >> ## test results in: >> "/tmp/haregtests-2020-01-17_14-01-45.SGkYcJ/vtc.12807.6adfff44" >> h1 haproxy h1 PID file check failed: >> h1 Bad exit status: 0x0100 exit 0x1 signal 0 core 0 >> Makefile:964: recipe for target 'reg-tests' failed >> make: *** [reg-tests] Error 1 >> > > git bisect blaims this commit: > > 25b569302167e71b32e569a2366027e8e320e80a is the first bad commit > commit 25b569302167e71b32e569a2366027e8e320e80a > Author: William Lallemand > Date: Tue Jan 14 15:38:43 2020 +0100 > > REGTEST: mcli/mcli_start_progs: start 2 programs > > This regtest tests the issue #446 by starting 2 programs and checking > if > they exist in the "show proc" of the master CLI. > > Should be backported as far as 2.0. > > > https://travis-ci.com/haproxy/haproxy is green > https://cirrus-ci.com/github/haproxy/haproxy is green > and what is even more interesting is that > https://travis-ci.org/martin-g/haproxy/builds (my fork with enabled ARM64 > testing on TravisCI) also just passed (after few failures due to timing > issues (I guess) > timing out might happen because of https://github.com/haproxy/haproxy/commit/ac8147446c7a3d1aa607042bc782095b03bc8dc4 your fork is 16 commits behind current master. try to rebase to master > > >> Regards, >> Martin >> >> On Fri, Jan 17, 2020 at 9:22 AM Илья Шипицин >> wrote: >> >>> привет! >>> >>> пт, 17 янв. 2020 г. в 11:42, Martin Grigorov >> >: >>> Привет Илья, On Thu, Jan 16, 2020 at 10:37 AM Илья Шипицин wrote: > > > чт, 16 янв. 2020 г. в 13:26, Martin Grigorov < > martin.grigo...@gmail.com>: > >> Hi Илья, >> >> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин >> wrote: >> >>> >>> Hello, Martin! >>> >>> btw, just curious, how is Apache Foundation (or you personally) >>> interested in all that ? >>> please do not blame me, I really like to know. >>> >> > ok, so you work in some company that is interested in haproxy on > ARM64. > you do not want to tell it's name, at least is it legal ? is it > related to some government ? > if "no" and "no", I guess most people won't ask any more questions :) > It is legal and I do not work for a government of any country! > > personally, I do not work at Haproxy Inc, I use haproxy, sometimes I > contribute to it. > Please do not consider me as an "official representative". > > > I'm interested in testing haproxy on ARM64, I planned to do so. I can > priorierize it according to your interest to it. > and yes, I can accept hardware donation (even considering I'm not part > of Haproxy Inc). > > also, from my point of view, what would be really useful in your case > is testing not just official reg-tests, but also > your configs. reg-tests cover only partially. If you enable clang asan > in your own workload there's a chance to catch > something interesting (or, to make sure your own workload is ok). I > can help with that as well. > Thanks for the offer! I've discussed it with my managers. Our offer is to donate a VM that could be used as an official CI agent for the HAProxy project long term. >>> >>> I'd split this into short term and long term approach. >>> >>> if you need to start any time soon, I'd focus on your own workload >>> testing first. I would build stand which emulates >>> your production workload, compile haproxy using clang address sanitizer >>> and give it a try (functional testing, load testing, ...) >>> >>> I can help with that. >>> >>> As for long term solution, currently haproxy simply cannot attach any >>> dedicated build agent to their CI. travis-ci does not allow >>> attaching dedicated agents. And haproxy team is very conservative in >>> adding new CI servers. >>> >>> I think, I will add arm64 (most probably openssl-1.1.1 only for now) >>> soon. Also, I'm going to investigate your libressl failures. >>> >>> so, dedicated vm definetly will help in troubleshooting issues, for >>> manual builds. It would save bunch of time. I do not mind if you will >>> add my ssh to that vm. >>> >>> >>> also, I requested access to Linaro. >>> >>> You can use Linaro for short term testing though. https://www.linaro.cloud/ Here you can request free VM for short periods:
Re: ARM(64) builds
On Fri, Jan 17, 2020 at 04:33:22PM +0200, Martin Grigorov wrote: > > git bisect blaims this commit: > > 25b569302167e71b32e569a2366027e8e320e80a is the first bad commit > commit 25b569302167e71b32e569a2366027e8e320e80a > Author: William Lallemand > Date: Tue Jan 14 15:38:43 2020 +0100 > > REGTEST: mcli/mcli_start_progs: start 2 programs Well that's the commit which introduces the vtc file so that's normal. > > https://travis-ci.com/haproxy/haproxy is green > https://cirrus-ci.com/github/haproxy/haproxy is green > and what is even more interesting is that > https://travis-ci.org/martin-g/haproxy/builds (my fork with enabled ARM64 > testing on TravisCI) also just passed (after few failures due to timing > issues (I guess) I don't see anything regarding mcli_start_progs.vtc in this links, can you provide the output of `make reg-tests -- --debug reg-tests/mcli/mcli_start_progs.vtc` ? Thanks -- William Lallemand
Re: ARM(64) builds
On Fri, Jan 17, 2020 at 4:13 PM Martin Grigorov wrote: > Hi all, > > Today's build consistently fails on my ARM64 VM: > > ## Starting vtest ## > Testing with haproxy version: 2.2-dev0-70c5b0-123 > #top TEST reg-tests/mcli/mcli_start_progs.vtc FAILED (3.004) exit=2 > 1 tests failed, 0 tests skipped, 34 tests passed > ## Gathering results ## > ## Test case: reg-tests/mcli/mcli_start_progs.vtc ## > ## test results in: > "/tmp/haregtests-2020-01-17_14-01-45.SGkYcJ/vtc.12807.6adfff44" > h1 haproxy h1 PID file check failed: > h1 Bad exit status: 0x0100 exit 0x1 signal 0 core 0 > Makefile:964: recipe for target 'reg-tests' failed > make: *** [reg-tests] Error 1 > git bisect blaims this commit: 25b569302167e71b32e569a2366027e8e320e80a is the first bad commit commit 25b569302167e71b32e569a2366027e8e320e80a Author: William Lallemand Date: Tue Jan 14 15:38:43 2020 +0100 REGTEST: mcli/mcli_start_progs: start 2 programs This regtest tests the issue #446 by starting 2 programs and checking if they exist in the "show proc" of the master CLI. Should be backported as far as 2.0. https://travis-ci.com/haproxy/haproxy is green https://cirrus-ci.com/github/haproxy/haproxy is green and what is even more interesting is that https://travis-ci.org/martin-g/haproxy/builds (my fork with enabled ARM64 testing on TravisCI) also just passed (after few failures due to timing issues (I guess) > Regards, > Martin > > On Fri, Jan 17, 2020 at 9:22 AM Илья Шипицин wrote: > >> привет! >> >> пт, 17 янв. 2020 г. в 11:42, Martin Grigorov : >> >>> Привет Илья, >>> >>> On Thu, Jan 16, 2020 at 10:37 AM Илья Шипицин >>> wrote: >>> чт, 16 янв. 2020 г. в 13:26, Martin Grigorov >>> >: > Hi Илья, > > On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин > wrote: > >> >> Hello, Martin! >> >> btw, just curious, how is Apache Foundation (or you personally) >> interested in all that ? >> please do not blame me, I really like to know. >> > ok, so you work in some company that is interested in haproxy on ARM64. you do not want to tell it's name, at least is it legal ? is it related to some government ? if "no" and "no", I guess most people won't ask any more questions :) >>> >>> It is legal and I do not work for a government of any country! >>> >>> personally, I do not work at Haproxy Inc, I use haproxy, sometimes I contribute to it. Please do not consider me as an "official representative". I'm interested in testing haproxy on ARM64, I planned to do so. I can priorierize it according to your interest to it. and yes, I can accept hardware donation (even considering I'm not part of Haproxy Inc). also, from my point of view, what would be really useful in your case is testing not just official reg-tests, but also your configs. reg-tests cover only partially. If you enable clang asan in your own workload there's a chance to catch something interesting (or, to make sure your own workload is ok). I can help with that as well. >>> >>> Thanks for the offer! >>> I've discussed it with my managers. >>> Our offer is to donate a VM that could be used as an official CI agent >>> for the HAProxy project long term. >>> >> >> I'd split this into short term and long term approach. >> >> if you need to start any time soon, I'd focus on your own workload >> testing first. I would build stand which emulates >> your production workload, compile haproxy using clang address sanitizer >> and give it a try (functional testing, load testing, ...) >> >> I can help with that. >> >> As for long term solution, currently haproxy simply cannot attach any >> dedicated build agent to their CI. travis-ci does not allow >> attaching dedicated agents. And haproxy team is very conservative in >> adding new CI servers. >> >> I think, I will add arm64 (most probably openssl-1.1.1 only for now) >> soon. Also, I'm going to investigate your libressl failures. >> >> so, dedicated vm definetly will help in troubleshooting issues, for >> manual builds. It would save bunch of time. I do not mind if you will >> add my ssh to that vm. >> >> >> also, I requested access to Linaro. >> >> >>> >>> You can use Linaro for short term testing though. >>> https://www.linaro.cloud/ >>> Here you can request free VM for short periods: >>> https://servicedesk.linaro.org/servicedesk/customer/portal/19/create/265 >>> P.S. Linaro is not my employer! >>> >>> >>> Regards, >>> Martin >>> >>> > >> > @apache.org is just one of my several emails. And it is the default > one in my email client. > ASF is not related anyhow to my participation here. > If I used my work email then it might look like some kind of > advertisement. I'd like to avoid that! > Next time I
Re: ARM(64) builds
Hi all, Today's build consistently fails on my ARM64 VM: ## Starting vtest ## Testing with haproxy version: 2.2-dev0-70c5b0-123 #top TEST reg-tests/mcli/mcli_start_progs.vtc FAILED (3.004) exit=2 1 tests failed, 0 tests skipped, 34 tests passed ## Gathering results ## ## Test case: reg-tests/mcli/mcli_start_progs.vtc ## ## test results in: "/tmp/haregtests-2020-01-17_14-01-45.SGkYcJ/vtc.12807.6adfff44" h1 haproxy h1 PID file check failed: h1 Bad exit status: 0x0100 exit 0x1 signal 0 core 0 Makefile:964: recipe for target 'reg-tests' failed make: *** [reg-tests] Error 1 Regards, Martin On Fri, Jan 17, 2020 at 9:22 AM Илья Шипицин wrote: > привет! > > пт, 17 янв. 2020 г. в 11:42, Martin Grigorov : > >> Привет Илья, >> >> On Thu, Jan 16, 2020 at 10:37 AM Илья Шипицин >> wrote: >> >>> >>> >>> чт, 16 янв. 2020 г. в 13:26, Martin Grigorov >> >: >>> Hi Илья, On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин wrote: > > Hello, Martin! > > btw, just curious, how is Apache Foundation (or you personally) > interested in all that ? > please do not blame me, I really like to know. > >>> ok, so you work in some company that is interested in haproxy on ARM64. >>> you do not want to tell it's name, at least is it legal ? is it related >>> to some government ? >>> if "no" and "no", I guess most people won't ask any more questions :) >>> >> >> It is legal and I do not work for a government of any country! >> >> >>> >>> personally, I do not work at Haproxy Inc, I use haproxy, sometimes I >>> contribute to it. >>> Please do not consider me as an "official representative". >>> >>> >>> I'm interested in testing haproxy on ARM64, I planned to do so. I can >>> priorierize it according to your interest to it. >>> and yes, I can accept hardware donation (even considering I'm not part >>> of Haproxy Inc). >>> >>> also, from my point of view, what would be really useful in your case is >>> testing not just official reg-tests, but also >>> your configs. reg-tests cover only partially. If you enable clang asan >>> in your own workload there's a chance to catch >>> something interesting (or, to make sure your own workload is ok). I can >>> help with that as well. >>> >> >> Thanks for the offer! >> I've discussed it with my managers. >> Our offer is to donate a VM that could be used as an official CI agent >> for the HAProxy project long term. >> > > I'd split this into short term and long term approach. > > if you need to start any time soon, I'd focus on your own workload testing > first. I would build stand which emulates > your production workload, compile haproxy using clang address sanitizer > and give it a try (functional testing, load testing, ...) > > I can help with that. > > As for long term solution, currently haproxy simply cannot attach any > dedicated build agent to their CI. travis-ci does not allow > attaching dedicated agents. And haproxy team is very conservative in > adding new CI servers. > > I think, I will add arm64 (most probably openssl-1.1.1 only for now) soon. > Also, I'm going to investigate your libressl failures. > > so, dedicated vm definetly will help in troubleshooting issues, for manual > builds. It would save bunch of time. I do not mind if you will > add my ssh to that vm. > > > also, I requested access to Linaro. > > >> >> You can use Linaro for short term testing though. >> https://www.linaro.cloud/ >> Here you can request free VM for short periods: >> https://servicedesk.linaro.org/servicedesk/customer/portal/19/create/265 >> P.S. Linaro is not my employer! >> >> >> Regards, >> Martin >> >> >>> > @apache.org is just one of my several emails. And it is the default one in my email client. ASF is not related anyhow to my participation here. If I used my work email then it might look like some kind of advertisement. I'd like to avoid that! Next time I will use my @gmail.com one, as more neutral. Actually I've used the GMail one when registering to this mailing list, so probably the post from @apache has been moderated. I'll be more careful next time! Thanks, Илья! Regards, Martin > > чт, 16 янв. 2020 г. в 12:32, Martin Grigorov : > >> Hello HAProxy developers, >> >> At work we are going to use more and more ARM64 based servers and >> HAProxy is one of the main products we rely on. >> So I went checking whether HAProxy project has a CI environment for >> testing on ARM architecture. >> > > we are looking towards > https://docs.travis-ci.com/user/multi-cpu-architectures > > >> I've found this recent discussion: >> https://www.mail-archive.com/haproxy@formilux.org/msg35302.html (I >> didn't find a way how to continue on the same mail thread, so I'm >> starting
Re: ARM(64) builds
привет! пт, 17 янв. 2020 г. в 11:42, Martin Grigorov : > Привет Илья, > > On Thu, Jan 16, 2020 at 10:37 AM Илья Шипицин > wrote: > >> >> >> чт, 16 янв. 2020 г. в 13:26, Martin Grigorov : >> >>> Hi Илья, >>> >>> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин >>> wrote: >>> Hello, Martin! btw, just curious, how is Apache Foundation (or you personally) interested in all that ? please do not blame me, I really like to know. >>> >> ok, so you work in some company that is interested in haproxy on ARM64. >> you do not want to tell it's name, at least is it legal ? is it related >> to some government ? >> if "no" and "no", I guess most people won't ask any more questions :) >> > > It is legal and I do not work for a government of any country! > > >> >> personally, I do not work at Haproxy Inc, I use haproxy, sometimes I >> contribute to it. >> Please do not consider me as an "official representative". >> >> >> I'm interested in testing haproxy on ARM64, I planned to do so. I can >> priorierize it according to your interest to it. >> and yes, I can accept hardware donation (even considering I'm not part of >> Haproxy Inc). >> >> also, from my point of view, what would be really useful in your case is >> testing not just official reg-tests, but also >> your configs. reg-tests cover only partially. If you enable clang asan in >> your own workload there's a chance to catch >> something interesting (or, to make sure your own workload is ok). I can >> help with that as well. >> > > Thanks for the offer! > I've discussed it with my managers. > Our offer is to donate a VM that could be used as an official CI agent for > the HAProxy project long term. > I'd split this into short term and long term approach. if you need to start any time soon, I'd focus on your own workload testing first. I would build stand which emulates your production workload, compile haproxy using clang address sanitizer and give it a try (functional testing, load testing, ...) I can help with that. As for long term solution, currently haproxy simply cannot attach any dedicated build agent to their CI. travis-ci does not allow attaching dedicated agents. And haproxy team is very conservative in adding new CI servers. I think, I will add arm64 (most probably openssl-1.1.1 only for now) soon. Also, I'm going to investigate your libressl failures. so, dedicated vm definetly will help in troubleshooting issues, for manual builds. It would save bunch of time. I do not mind if you will add my ssh to that vm. also, I requested access to Linaro. > > You can use Linaro for short term testing though. > https://www.linaro.cloud/ > Here you can request free VM for short periods: > https://servicedesk.linaro.org/servicedesk/customer/portal/19/create/265 > P.S. Linaro is not my employer! > > > Regards, > Martin > > >> >>> >>> @apache.org is just one of my several emails. And it is the default one >>> in my email client. >>> ASF is not related anyhow to my participation here. >>> If I used my work email then it might look like some kind of >>> advertisement. I'd like to avoid that! >>> Next time I will use my @gmail.com one, as more neutral. Actually I've >>> used the GMail one when registering to this mailing list, so probably the >>> post from @apache has been moderated. I'll be more careful next time! >>> >>> Thanks, Илья! >>> >>> Regards, >>> Martin >>> >>> чт, 16 янв. 2020 г. в 12:32, Martin Grigorov : > Hello HAProxy developers, > > At work we are going to use more and more ARM64 based servers and > HAProxy is one of the main products we rely on. > So I went checking whether HAProxy project has a CI environment for > testing on ARM architecture. > we are looking towards https://docs.travis-ci.com/user/multi-cpu-architectures > I've found this recent discussion: > https://www.mail-archive.com/haproxy@formilux.org/msg35302.html (I > didn't find a way how to continue on the same mail thread, so I'm starting > a new one. Apologies for that!). > I played with arm64 for a while, the issue I encountered is travis-ci cache key, i.e. we cache openssl builds between our builds. so travis used the same cache key for both amd64 and arm64 builds (this might have changed recently, I did not check yet) arm64 is in my queue (as well as recent s390x arch from travis), hope to get back to it within month or so. > From this discussion and from > https://github.com/haproxy/haproxy/blob/master/.travis.yml I > understand that there is no public CI in use (i.e. TravisCI or CirrusCI) > but some of the developers run some tests locally regularly. > it is not completely true. there's public CI. we do not use github PR machinery, so sometimes tests fail after push to master branch. it is considered as ok, failures are fixed pretty fast. for
Re: ARM(64) builds
Привет Илья, On Thu, Jan 16, 2020 at 10:37 AM Илья Шипицин wrote: > > > чт, 16 янв. 2020 г. в 13:26, Martin Grigorov : > >> Hi Илья, >> >> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин >> wrote: >> >>> >>> Hello, Martin! >>> >>> btw, just curious, how is Apache Foundation (or you personally) >>> interested in all that ? >>> please do not blame me, I really like to know. >>> >> > ok, so you work in some company that is interested in haproxy on ARM64. > you do not want to tell it's name, at least is it legal ? is it related to > some government ? > if "no" and "no", I guess most people won't ask any more questions :) > It is legal and I do not work for a government of any country! > > personally, I do not work at Haproxy Inc, I use haproxy, sometimes I > contribute to it. > Please do not consider me as an "official representative". > > > I'm interested in testing haproxy on ARM64, I planned to do so. I can > priorierize it according to your interest to it. > and yes, I can accept hardware donation (even considering I'm not part of > Haproxy Inc). > > also, from my point of view, what would be really useful in your case is > testing not just official reg-tests, but also > your configs. reg-tests cover only partially. If you enable clang asan in > your own workload there's a chance to catch > something interesting (or, to make sure your own workload is ok). I can > help with that as well. > Thanks for the offer! I've discussed it with my managers. Our offer is to donate a VM that could be used as an official CI agent for the HAProxy project long term. You can use Linaro for short term testing though. https://www.linaro.cloud/ Here you can request free VM for short periods: https://servicedesk.linaro.org/servicedesk/customer/portal/19/create/265 P.S. Linaro is not my employer! Regards, Martin > >> >>> >> @apache.org is just one of my several emails. And it is the default one >> in my email client. >> ASF is not related anyhow to my participation here. >> If I used my work email then it might look like some kind of >> advertisement. I'd like to avoid that! >> Next time I will use my @gmail.com one, as more neutral. Actually I've >> used the GMail one when registering to this mailing list, so probably the >> post from @apache has been moderated. I'll be more careful next time! >> >> Thanks, Илья! >> >> Regards, >> Martin >> >> >>> >>> чт, 16 янв. 2020 г. в 12:32, Martin Grigorov : >>> Hello HAProxy developers, At work we are going to use more and more ARM64 based servers and HAProxy is one of the main products we rely on. So I went checking whether HAProxy project has a CI environment for testing on ARM architecture. >>> >>> we are looking towards >>> https://docs.travis-ci.com/user/multi-cpu-architectures >>> >>> I've found this recent discussion: https://www.mail-archive.com/haproxy@formilux.org/msg35302.html (I didn't find a way how to continue on the same mail thread, so I'm starting a new one. Apologies for that!). >>> >>> I played with arm64 for a while, the issue I encountered is travis-ci >>> cache key, i.e. we cache openssl builds between our builds. >>> so travis used the same cache key for both amd64 and arm64 builds (this >>> might have changed recently, I did not check yet) >>> >>> arm64 is in my queue (as well as recent s390x arch from travis), hope to >>> get back to it within month or so. >>> >>> From this discussion and from https://github.com/haproxy/haproxy/blob/master/.travis.yml I understand that there is no public CI in use (i.e. TravisCI or CirrusCI) but some of the developers run some tests locally regularly. >>> >>> it is not completely true. >>> there's public CI. we do not use github PR machinery, so sometimes tests >>> fail after push to master branch. it is considered as ok, failures are >>> fixed pretty fast. >>> for example, see >>> https://www.mail-archive.com/haproxy@formilux.org/msg35910.html >>> it was just perfect, failure detected using CI and fixed within few >>> days. no customers affected. >>> >>> I've forked the project and tested on TravisCI ( https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the builds were not very stable: 1) some tests fail sometimes. I guess it is because of some timing issues For example: - https://travis-ci.org/martin-g/haproxy/jobs/636745241 - https://travis-ci.org/martin-g/haproxy/jobs/636750676 - https://travis-ci.org/martin-g/haproxy/jobs/636763346 >>> >>> that's very interesting. I'll have a look. >>> >>> >>> 2) There was some weird issue on testing with LibreSSL The output redirect at https://github.com/haproxy/haproxy/blob/bb9da0b8e23c46caab34fe6005b66fa8065fe3ac/.travis.yml#L96 for some reason got stuck the build. I've removed temporarily the output redirects and then it passed. So, it looks like some issue with TravisCI environment. >>>
Re: ARM(64) builds
чт, 16 янв. 2020 г. в 13:37, Илья Шипицин : > > > чт, 16 янв. 2020 г. в 13:26, Martin Grigorov : > >> Hi Илья, >> >> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин >> wrote: >> >>> >>> Hello, Martin! >>> >>> btw, just curious, how is Apache Foundation (or you personally) >>> interested in all that ? >>> please do not blame me, I really like to know. >>> >> > ok, so you work in some company that is interested in haproxy on ARM64. > you do not want to tell it's name, at least is it legal ? is it related to > some government ? > if "no" and "no", I guess most people won't ask any more questions :) > oops. I meant "yes" and "no" > > personally, I do not work at Haproxy Inc, I use haproxy, sometimes I > contribute to it. > Please do not consider me as an "official representative". > > > I'm interested in testing haproxy on ARM64, I planned to do so. I can > priorierize it according to your interest to it. > and yes, I can accept hardware donation (even considering I'm not part of > Haproxy Inc). > > also, from my point of view, what would be really useful in your case is > testing not just official reg-tests, but also > your configs. reg-tests cover only partially. If you enable clang asan in > your own workload there's a chance to catch > something interesting (or, to make sure your own workload is ok). I can > help with that as well. > > >> >>> >> @apache.org is just one of my several emails. And it is the default one >> in my email client. >> ASF is not related anyhow to my participation here. >> If I used my work email then it might look like some kind of >> advertisement. I'd like to avoid that! >> Next time I will use my @gmail.com one, as more neutral. Actually I've >> used the GMail one when registering to this mailing list, so probably the >> post from @apache has been moderated. I'll be more careful next time! >> >> Thanks, Илья! >> >> Regards, >> Martin >> >> >>> >>> чт, 16 янв. 2020 г. в 12:32, Martin Grigorov : >>> Hello HAProxy developers, At work we are going to use more and more ARM64 based servers and HAProxy is one of the main products we rely on. So I went checking whether HAProxy project has a CI environment for testing on ARM architecture. >>> >>> we are looking towards >>> https://docs.travis-ci.com/user/multi-cpu-architectures >>> >>> I've found this recent discussion: https://www.mail-archive.com/haproxy@formilux.org/msg35302.html (I didn't find a way how to continue on the same mail thread, so I'm starting a new one. Apologies for that!). >>> >>> I played with arm64 for a while, the issue I encountered is travis-ci >>> cache key, i.e. we cache openssl builds between our builds. >>> so travis used the same cache key for both amd64 and arm64 builds (this >>> might have changed recently, I did not check yet) >>> >>> arm64 is in my queue (as well as recent s390x arch from travis), hope to >>> get back to it within month or so. >>> >>> From this discussion and from https://github.com/haproxy/haproxy/blob/master/.travis.yml I understand that there is no public CI in use (i.e. TravisCI or CirrusCI) but some of the developers run some tests locally regularly. >>> >>> it is not completely true. >>> there's public CI. we do not use github PR machinery, so sometimes tests >>> fail after push to master branch. it is considered as ok, failures are >>> fixed pretty fast. >>> for example, see >>> https://www.mail-archive.com/haproxy@formilux.org/msg35910.html >>> it was just perfect, failure detected using CI and fixed within few >>> days. no customers affected. >>> >>> I've forked the project and tested on TravisCI ( https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the builds were not very stable: 1) some tests fail sometimes. I guess it is because of some timing issues For example: - https://travis-ci.org/martin-g/haproxy/jobs/636745241 - https://travis-ci.org/martin-g/haproxy/jobs/636750676 - https://travis-ci.org/martin-g/haproxy/jobs/636763346 >>> >>> that's very interesting. I'll have a look. >>> >>> >>> 2) There was some weird issue on testing with LibreSSL The output redirect at https://github.com/haproxy/haproxy/blob/bb9da0b8e23c46caab34fe6005b66fa8065fe3ac/.travis.yml#L96 for some reason got stuck the build. I've removed temporarily the output redirects and then it passed. So, it looks like some issue with TravisCI environment. >>> >>> arm64 is slower, I guess we should add "*travis*_*wait* *30" *to >>> build-ssl.sh script >>> thank for the hint >>> >>> In addition I've run the build and tests on one of our machines and all was OK! My question to you is: Are you happy with your current way of testing ARM architectures or you want to add more ? Here are some options: 1) enable TravisCI >>> >>> already done >>> >>> https://travis-ci.com/haproxy/haproxy
Re: ARM(64) builds
чт, 16 янв. 2020 г. в 13:26, Martin Grigorov : > Hi Илья, > > On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин > wrote: > >> >> Hello, Martin! >> >> btw, just curious, how is Apache Foundation (or you personally) >> interested in all that ? >> please do not blame me, I really like to know. >> > ok, so you work in some company that is interested in haproxy on ARM64. you do not want to tell it's name, at least is it legal ? is it related to some government ? if "no" and "no", I guess most people won't ask any more questions :) personally, I do not work at Haproxy Inc, I use haproxy, sometimes I contribute to it. Please do not consider me as an "official representative". I'm interested in testing haproxy on ARM64, I planned to do so. I can priorierize it according to your interest to it. and yes, I can accept hardware donation (even considering I'm not part of Haproxy Inc). also, from my point of view, what would be really useful in your case is testing not just official reg-tests, but also your configs. reg-tests cover only partially. If you enable clang asan in your own workload there's a chance to catch something interesting (or, to make sure your own workload is ok). I can help with that as well. > >> > @apache.org is just one of my several emails. And it is the default one > in my email client. > ASF is not related anyhow to my participation here. > If I used my work email then it might look like some kind of > advertisement. I'd like to avoid that! > Next time I will use my @gmail.com one, as more neutral. Actually I've > used the GMail one when registering to this mailing list, so probably the > post from @apache has been moderated. I'll be more careful next time! > > Thanks, Илья! > > Regards, > Martin > > >> >> чт, 16 янв. 2020 г. в 12:32, Martin Grigorov : >> >>> Hello HAProxy developers, >>> >>> At work we are going to use more and more ARM64 based servers and >>> HAProxy is one of the main products we rely on. >>> So I went checking whether HAProxy project has a CI environment for >>> testing on ARM architecture. >>> >> >> we are looking towards >> https://docs.travis-ci.com/user/multi-cpu-architectures >> >> >>> I've found this recent discussion: >>> https://www.mail-archive.com/haproxy@formilux.org/msg35302.html (I >>> didn't find a way how to continue on the same mail thread, so I'm starting >>> a new one. Apologies for that!). >>> >> >> I played with arm64 for a while, the issue I encountered is travis-ci >> cache key, i.e. we cache openssl builds between our builds. >> so travis used the same cache key for both amd64 and arm64 builds (this >> might have changed recently, I did not check yet) >> >> arm64 is in my queue (as well as recent s390x arch from travis), hope to >> get back to it within month or so. >> >> >>> From this discussion and from >>> https://github.com/haproxy/haproxy/blob/master/.travis.yml I understand >>> that there is no public CI in use (i.e. TravisCI or CirrusCI) but some of >>> the developers run some tests locally regularly. >>> >> >> it is not completely true. >> there's public CI. we do not use github PR machinery, so sometimes tests >> fail after push to master branch. it is considered as ok, failures are >> fixed pretty fast. >> for example, see >> https://www.mail-archive.com/haproxy@formilux.org/msg35910.html >> it was just perfect, failure detected using CI and fixed within few days. >> no customers affected. >> >> >>> I've forked the project and tested on TravisCI ( >>> https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the >>> builds were not very stable: >>> 1) some tests fail sometimes. I guess it is because of some timing issues >>> For example: >>> - https://travis-ci.org/martin-g/haproxy/jobs/636745241 >>> - https://travis-ci.org/martin-g/haproxy/jobs/636750676 >>> - https://travis-ci.org/martin-g/haproxy/jobs/636763346 >>> >> >> that's very interesting. I'll have a look. >> >> >> >>> 2) There was some weird issue on testing with LibreSSL >>> The output redirect at >>> https://github.com/haproxy/haproxy/blob/bb9da0b8e23c46caab34fe6005b66fa8065fe3ac/.travis.yml#L96 >>> for >>> some reason got stuck the build. I've removed temporarily the output >>> redirects and then it passed. So, it looks like some issue with TravisCI >>> environment. >>> >> >> arm64 is slower, I guess we should add "*travis*_*wait* *30" *to >> build-ssl.sh script >> thank for the hint >> >> >>> >>> In addition I've run the build and tests on one of our machines and all >>> was OK! >>> >>> My question to you is: Are you happy with your current way of testing >>> ARM architectures or you want to add more ? >>> Here are some options: >>> 1) enable TravisCI >>> >> >> already done >> >> https://travis-ci.com/haproxy/haproxy >> >> >>> 2) my company is willing to donate an ARM64 based VM, if you are >>> interested. >>> >> >> I do not work at Haproxy Inc :) >> Willy ? >> >> >>> You will have a SSH access and a user with sudo permissions to install >>> anything that is
Re: ARM(64) builds
Hi Илья, On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин wrote: > > Hello, Martin! > > btw, just curious, how is Apache Foundation (or you personally) interested > in all that ? > please do not blame me, I really like to know. > > @apache.org is just one of my several emails. And it is the default one in my email client. ASF is not related anyhow to my participation here. If I used my work email then it might look like some kind of advertisement. I'd like to avoid that! Next time I will use my @gmail.com one, as more neutral. Actually I've used the GMail one when registering to this mailing list, so probably the post from @apache has been moderated. I'll be more careful next time! Thanks, Илья! Regards, Martin > > чт, 16 янв. 2020 г. в 12:32, Martin Grigorov : > >> Hello HAProxy developers, >> >> At work we are going to use more and more ARM64 based servers and HAProxy >> is one of the main products we rely on. >> So I went checking whether HAProxy project has a CI environment for >> testing on ARM architecture. >> > > we are looking towards > https://docs.travis-ci.com/user/multi-cpu-architectures > > >> I've found this recent discussion: >> https://www.mail-archive.com/haproxy@formilux.org/msg35302.html (I >> didn't find a way how to continue on the same mail thread, so I'm starting >> a new one. Apologies for that!). >> > > I played with arm64 for a while, the issue I encountered is travis-ci > cache key, i.e. we cache openssl builds between our builds. > so travis used the same cache key for both amd64 and arm64 builds (this > might have changed recently, I did not check yet) > > arm64 is in my queue (as well as recent s390x arch from travis), hope to > get back to it within month or so. > > >> From this discussion and from >> https://github.com/haproxy/haproxy/blob/master/.travis.yml I understand >> that there is no public CI in use (i.e. TravisCI or CirrusCI) but some of >> the developers run some tests locally regularly. >> > > it is not completely true. > there's public CI. we do not use github PR machinery, so sometimes tests > fail after push to master branch. it is considered as ok, failures are > fixed pretty fast. > for example, see > https://www.mail-archive.com/haproxy@formilux.org/msg35910.html > it was just perfect, failure detected using CI and fixed within few days. > no customers affected. > > >> I've forked the project and tested on TravisCI ( >> https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the >> builds were not very stable: >> 1) some tests fail sometimes. I guess it is because of some timing issues >> For example: >> - https://travis-ci.org/martin-g/haproxy/jobs/636745241 >> - https://travis-ci.org/martin-g/haproxy/jobs/636750676 >> - https://travis-ci.org/martin-g/haproxy/jobs/636763346 >> > > that's very interesting. I'll have a look. > > > >> 2) There was some weird issue on testing with LibreSSL >> The output redirect at >> https://github.com/haproxy/haproxy/blob/bb9da0b8e23c46caab34fe6005b66fa8065fe3ac/.travis.yml#L96 >> for >> some reason got stuck the build. I've removed temporarily the output >> redirects and then it passed. So, it looks like some issue with TravisCI >> environment. >> > > arm64 is slower, I guess we should add "*travis*_*wait* *30" *to > build-ssl.sh script > thank for the hint > > >> >> In addition I've run the build and tests on one of our machines and all >> was OK! >> >> My question to you is: Are you happy with your current way of testing ARM >> architectures or you want to add more ? >> Here are some options: >> 1) enable TravisCI >> > > already done > > https://travis-ci.com/haproxy/haproxy > > >> 2) my company is willing to donate an ARM64 based VM, if you are >> interested. >> > > I do not work at Haproxy Inc :) > Willy ? > > >> You will have a SSH access and a user with sudo permissions to install >> anything that is needed. >> The spec is aarch64 8 cores CPU 2GHz (Kunpeng), 16GB RAM, 500G disk space >> and 5M network bandwidth. The OS could be any of CentOS 7.4/7.5/7.6, >> EulerOS 2.8, Fedora 29, OpenSuse 15.0 and Ubuntu 16.04/18.04. >> >> In both cases it will be ARM64. From the earlier mail discussion I >> understand you would prefer ARM32. >> > > as for myself, I prefer both arm64 and arm32. > however, both AMD64 and ARM64 are the same 64 bits. both of them are > little-endian. but you mentioned at least 3 builds with ARM64 failing (we > have those tests passing on AMD64)! > > > >> >> Kind regards, >> Martin >> >
Re: ARM(64) builds
Hello, Martin! btw, just curious, how is Apache Foundation (or you personally) interested in all that ? please do not blame me, I really like to know. чт, 16 янв. 2020 г. в 12:32, Martin Grigorov : > Hello HAProxy developers, > > At work we are going to use more and more ARM64 based servers and HAProxy > is one of the main products we rely on. > So I went checking whether HAProxy project has a CI environment for > testing on ARM architecture. > we are looking towards https://docs.travis-ci.com/user/multi-cpu-architectures > I've found this recent discussion: > https://www.mail-archive.com/haproxy@formilux.org/msg35302.html (I > didn't find a way how to continue on the same mail thread, so I'm starting > a new one. Apologies for that!). > I played with arm64 for a while, the issue I encountered is travis-ci cache key, i.e. we cache openssl builds between our builds. so travis used the same cache key for both amd64 and arm64 builds (this might have changed recently, I did not check yet) arm64 is in my queue (as well as recent s390x arch from travis), hope to get back to it within month or so. > From this discussion and from > https://github.com/haproxy/haproxy/blob/master/.travis.yml I understand > that there is no public CI in use (i.e. TravisCI or CirrusCI) but some of > the developers run some tests locally regularly. > it is not completely true. there's public CI. we do not use github PR machinery, so sometimes tests fail after push to master branch. it is considered as ok, failures are fixed pretty fast. for example, see https://www.mail-archive.com/haproxy@formilux.org/msg35910.html it was just perfect, failure detected using CI and fixed within few days. no customers affected. > I've forked the project and tested on TravisCI ( > https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the > builds were not very stable: > 1) some tests fail sometimes. I guess it is because of some timing issues > For example: > - https://travis-ci.org/martin-g/haproxy/jobs/636745241 > - https://travis-ci.org/martin-g/haproxy/jobs/636750676 > - https://travis-ci.org/martin-g/haproxy/jobs/636763346 > that's very interesting. I'll have a look. > 2) There was some weird issue on testing with LibreSSL > The output redirect at > https://github.com/haproxy/haproxy/blob/bb9da0b8e23c46caab34fe6005b66fa8065fe3ac/.travis.yml#L96 > for > some reason got stuck the build. I've removed temporarily the output > redirects and then it passed. So, it looks like some issue with TravisCI > environment. > arm64 is slower, I guess we should add "*travis*_*wait* *30" *to build-ssl.sh script thank for the hint > > In addition I've run the build and tests on one of our machines and all > was OK! > > My question to you is: Are you happy with your current way of testing ARM > architectures or you want to add more ? > Here are some options: > 1) enable TravisCI > already done https://travis-ci.com/haproxy/haproxy > 2) my company is willing to donate an ARM64 based VM, if you are > interested. > I do not work at Haproxy Inc :) Willy ? > You will have a SSH access and a user with sudo permissions to install > anything that is needed. > The spec is aarch64 8 cores CPU 2GHz (Kunpeng), 16GB RAM, 500G disk space > and 5M network bandwidth. The OS could be any of CentOS 7.4/7.5/7.6, > EulerOS 2.8, Fedora 29, OpenSuse 15.0 and Ubuntu 16.04/18.04. > > In both cases it will be ARM64. From the earlier mail discussion I > understand you would prefer ARM32. > as for myself, I prefer both arm64 and arm32. however, both AMD64 and ARM64 are the same 64 bits. both of them are little-endian. but you mentioned at least 3 builds with ARM64 failing (we have those tests passing on AMD64)! > > Kind regards, > Martin >
ARM(64) builds
Hello HAProxy developers, At work we are going to use more and more ARM64 based servers and HAProxy is one of the main products we rely on. So I went checking whether HAProxy project has a CI environment for testing on ARM architecture. I've found this recent discussion: https://www.mail-archive.com/haproxy@formilux.org/msg35302.html (I didn't find a way how to continue on the same mail thread, so I'm starting a new one. Apologies for that!). >From this discussion and from https://github.com/haproxy/haproxy/blob/master/.travis.yml I understand that there is no public CI in use (i.e. TravisCI or CirrusCI) but some of the developers run some tests locally regularly. I've forked the project and tested on TravisCI ( https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the builds were not very stable: 1) some tests fail sometimes. I guess it is because of some timing issues For example: - https://travis-ci.org/martin-g/haproxy/jobs/636745241 - https://travis-ci.org/martin-g/haproxy/jobs/636750676 - https://travis-ci.org/martin-g/haproxy/jobs/636763346 2) There was some weird issue on testing with LibreSSL The output redirect at https://github.com/haproxy/haproxy/blob/bb9da0b8e23c46caab34fe6005b66fa8065fe3ac/.travis.yml#L96 for some reason got stuck the build. I've removed temporarily the output redirects and then it passed. So, it looks like some issue with TravisCI environment. In addition I've run the build and tests on one of our machines and all was OK! My question to you is: Are you happy with your current way of testing ARM architectures or you want to add more ? Here are some options: 1) enable TravisCI 2) my company is willing to donate an ARM64 based VM, if you are interested. You will have a SSH access and a user with sudo permissions to install anything that is needed. The spec is aarch64 8 cores CPU 2GHz (Kunpeng), 16GB RAM, 500G disk space and 5M network bandwidth. The OS could be any of CentOS 7.4/7.5/7.6, EulerOS 2.8, Fedora 29, OpenSuse 15.0 and Ubuntu 16.04/18.04. In both cases it will be ARM64. From the earlier mail discussion I understand you would prefer ARM32. Kind regards, Martin
