Re: [2.0.17] crash with coredump

[Kirill A. Korinsky]

Hey,

I'm wondering, does it related to this code:

+   /* some tasks may have woken other ones up */
+   if (max_processed && thread_has_tasks())
+   goto not_done_yet;
+

from 
http://git.haproxy.org/?p=haproxy-2.2.git;a=blobdiff;f=src/task.c;h=500223f185bf324c0adb34a42ec0244e638ce63e;hp=1a7f44d9169e0a01d42ba13d8d335102aa43577b;hb=5c8be272c732e4f42ccd6b3d65f25aa7425a2aba;hpb=77015abe0bcfde67bff519b1d48393a513015f77
 
<http://git.haproxy.org/?p=haproxy-2.2.git;a=blobdiff;f=src/task.c;h=500223f185bf324c0adb34a42ec0244e638ce63e;hp=1a7f44d9169e0a01d42ba13d8d335102aa43577b;hb=5c8be272c732e4f42ccd6b3d65f25aa7425a2aba;hpb=77015abe0bcfde67bff519b1d48393a513015f77>

as far as I understand it should be safe to remove (with not_done_yet label).

Can you try it?

--
wbr, Kirill

> On 3. Nov 2020, at 15:15, Maciej Zdeb  wrote:
> 
> I modified h2s struct in 2.2 branch with HEAD set to 
> f96508aae6b49277dcf142caa35042678cf8e2ca "MEDIUM: mux-h2: merge recv_wait and 
> send_wait event notifications" like below (subs is in exact place of removed 
> wait_event):
> 
> struct h2s {
> [...]
> struct tasklet *dummy0;
> struct wait_event *dummy1;
> struct wait_event *subs;  /* recv wait_event the conn_stream 
> associated is waiting on (via h2_subscribe) */
> struct list list; /* To be used when adding in h2c->send_list or 
> h2c->fctl_lsit */
> struct tasklet *shut_tl;  /* deferred shutdown tasklet, to retry to 
> send an RST after we failed to,
>* in case there's no other subscription to 
> do it */
> }
> 
> it crashed like before with subs = 0x:
> 
> (gdb) p *(struct h2s*)(0x7fde7459e9b0)
> $1 = {cs = 0x7fde5c02d260, sess = 0x5628283bc740 , h2c = 
> 0x5628295cbb80, h1m = {state = H1_MSG_RPBEFORE, flags = 12, curr_len = 0,
> body_len = 0, next = 0, err_pos = -1, err_state = 0}, by_id = {node = 
> {branches = {b = {0x0, 0x7fde3c2c6c60}}, node_p = 0x0,
>   leaf_p = 0x5628295cc018, bit = -5624, pfx = 29785}, key = 11}, id = 11, 
> flags = 28673, sws = -4060, errcode = H2_ERR_NO_ERROR, st = H2_SS_HREM,
>   status = 200, body_len = 0, rxbuf = {size = 0, area = 0x0, data = 0, head = 
> 0}, dummy0 = 0x0, dummy1 = 0x0, subs = 0x, list = {
> n = 0x7fde7459ea68, p = 0x7fde7459ea68}, shut_tl = 0x5628297eeaf0}
> 
> it crashes like above until commit: 
> http://git.haproxy.org/?p=haproxy-2.2.git;a=commit;h=5c8be272c732e4f42ccd6b3d65f25aa7425a2aba
>  
> <http://git.haproxy.org/?p=haproxy-2.2.git;a=commit;h=5c8be272c732e4f42ccd6b3d65f25aa7425a2aba>
>  which alters tasks processing.
> 
> 
> pon., 2 lis 2020 o 15:46 Maciej Zdeb mailto:mac...@zdeb.pl>> 
> napisał(a):
> I'm wondering, the corrupted address was always at "wait_event" in h2s 
> struct, after its removal in: 
> http://git.haproxy.org/?p=haproxy-2.2.git;a=commitdiff;h=5723f295d85febf5505f8aef6afabb6b23d6fdec;hp=f11be0ea1e8e571234cb41a2fcdde2cf2161df37
>  
> <http://git.haproxy.org/?p=haproxy-2.2.git;a=commitdiff;h=5723f295d85febf5505f8aef6afabb6b23d6fdec;hp=f11be0ea1e8e571234cb41a2fcdde2cf2161df37>
>  crashes went away.
> 
> But with the above patch and after altering h2s struct into:
> struct h2s {
> [...]
> struct tasklet *shut_tl;
> struct wait_event *recv_wait; /* recv wait_event the conn_stream 
> associated is waiting on (via h2_subscribe) */
> struct wait_event *send_wait; /* send wait_event the conn_stream 
> associated is waiting on (via h2_subscribe) */
> struct list list; /* To be used when adding in h2c->send_list or 
> h2c->fctl_lsit */
> };
> 
> the crash returned.
> 
> However after recv_wait and send_wait were merged in: 
> http://git.haproxy.org/?p=haproxy-2.2.git;a=commit;h=f96508aae6b49277dcf142caa35042678cf8e2ca
>  
> <http://git.haproxy.org/?p=haproxy-2.2.git;a=commit;h=f96508aae6b49277dcf142caa35042678cf8e2ca>
>  crashes went away again.
> 
> In my opinion shut_tl should be corrupted again, but it is not. Maybe the 
> last patch fixed it?
> 
> pon., 2 lis 2020 o 15:37 Kirill A. Korinsky  <mailto:kir...@korins.ky>> napisał(a):
> Maciej,
> 
> Looks like memory corruption is still here but it corrupt just some another 
> place.
> 
> Willy do you agree?
> 
> --
> wbr, Kirill
> 
>> On 2. Nov 2020, at 15:34, Maciej Zdeb > <mailto:mac...@zdeb.pl>> wrote:
>> 
>> So after Kirill suggestion to modify h2s struct in a way that tasklet 
>> "shut_tl" is before recv_wait I verified if in 2.2.4 the same crash will 
>> occur nd it did not!
>> 
>> After the patch that merges recv_wait and send_wait: 
>&g

Re: [2.0.17] crash with coredump

[Kirill A. Korinsky]

Maciej,

Looks like memory corruption is still here but it corrupt just some another 
place.

Willy do you agree?

--
wbr, Kirill

> On 2. Nov 2020, at 15:34, Maciej Zdeb  wrote:
> 
> So after Kirill suggestion to modify h2s struct in a way that tasklet 
> "shut_tl" is before recv_wait I verified if in 2.2.4 the same crash will 
> occur nd it did not!
> 
> After the patch that merges recv_wait and send_wait: 
> http://git.haproxy.org/?p=haproxy-2.2.git;a=commit;h=f96508aae6b49277dcf142caa35042678cf8e2ca
>  
> <http://git.haproxy.org/?p=haproxy-2.2.git;a=commit;h=f96508aae6b49277dcf142caa35042678cf8e2ca>
> and witch such h2s (tasklet shut_tl before wait_event subs) the crashes are 
> gone:
> 
> struct h2s {
> [...]
> struct buffer rxbuf; /* receive buffer, always valid (buf_empty or 
> real buffer) */
> struct tasklet *shut_tl;  /* deferred shutdown tasklet, to retry to 
> send an RST after we failed to,
>* in case there's no other subscription to 
> do it */
> struct wait_event *subs;  /* recv wait_event the conn_stream 
> associated is waiting on (via h2_subscribe) */
> struct list list; /* To be used when adding in h2c->send_list or 
> h2c->fctl_lsit */
> };
> 
> 
> 
> pon., 2 lis 2020 o 12:42 Maciej Zdeb mailto:mac...@zdeb.pl>> 
> napisał(a):
> Great idea Kirill,
> 
> With such modification:
> 
> struct h2s {
> [...]
> struct tasklet *shut_tl;
> struct wait_event *recv_wait; /* recv wait_event the conn_stream 
> associated is waiting on (via h2_subscribe) */
> struct wait_event *send_wait; /* send wait_event the conn_stream 
> associated is waiting on (via h2_subscribe) */
>     struct list list; /* To be used when adding in h2c->send_list or 
> h2c->fctl_lsit */
> };
> 
> it crashed just like before.
> 
> pon., 2 lis 2020 o 11:12 Kirill A. Korinsky  <mailto:kir...@korins.ky>> napisał(a):
> Hi,
> 
> Thanks for update.
> 
> After read Wully's recommendation and provided commit that fixed an issue I'm 
> curious can you "edit" a bit this commit and move `shut_tl` before 
> `recv_wait` instead of removed `wait_event`?
> 
> It is a quiet dummy way to confirm that memory corruption had gone, and not 
> just moved to somewhere else.
> 
> --
> wbr, Kirill
> 
>> On 2. Nov 2020, at 10:58, Maciej Zdeb > <mailto:mac...@zdeb.pl>> wrote:
>> 
>> Hi,
>> 
>> Update for people on the list that might be interested in the issue, because 
>> part of discussion was private.
>> 
>> I wanted to check Willy suggestion and modified h2s struct (added dummy 
>> fields):
>> 
>> struct h2s {
>> [...]
>> uint16_t status; /* HTTP response status */
>> unsigned long long body_len; /* remaining body length according to 
>> content-length if H2_SF_DATA_CLEN */
>> struct buffer rxbuf; /* receive buffer, always valid (buf_empty or 
>> real buffer) */
>> int dummy0;
>> struct wait_event wait_event; /* Wait list, when we're attempting to 
>> send a RST but we can't send */
>> int dummy1;
>> struct wait_event *recv_wait; /* recv wait_event the conn_stream 
>> associated is waiting on (via h2_subscribe) */
>> int dummy2;
>> struct wait_event *send_wait; /* send wait_event the conn_stream 
>> associated is waiting on (via h2_subscribe) */
>> int dummy3;
>> struct list list; /* To be used when adding in h2c->send_list or 
>> h2c->fctl_lsit */
>> struct list sending_list; /* To be used when adding in 
>> h2c->sending_list */
>> };
>> 
>> With such modified h2s struct, the crash did not occur.
>> 
>> I've checked HAProxy 2.1, it crashes like 2.0.
>> 
>> I've also checked 2.2, bisection showed that this commit: 
>> http://git.haproxy.org/?p=haproxy-2.2.git;a=commitdiff;h=5723f295d85febf5505f8aef6afabb6b23d6fdec;hp=f11be0ea1e8e571234cb41a2fcdde2cf2161df37
>>  
>> <http://git.haproxy.org/?p=haproxy-2.2.git;a=commitdiff;h=5723f295d85febf5505f8aef6afabb6b23d6fdec;hp=f11be0ea1e8e571234cb41a2fcdde2cf2161df37>
>>  fixed the crashes we experienced. I'm not sure how/if it fixed the memory 
>> corruption, it is possible that memory is still corrupted but not causing 
>> the crash.
>> 
>> 
>> 
>> pt., 25 wrz 2020 o 16:25 Kirill A. Korinsky > <mailto:kir...@korins.ky>> napisał(a):
>> Very interesting.
>> 
>> Anyway, I can see that this pice of code was ref

Re: [2.0.17] crash with coredump

[Kirill A. Korinsky]

Hi,

Thanks for update.

After read Wully's recommendation and provided commit that fixed an issue I'm 
curious can you "edit" a bit this commit and move `shut_tl` before `recv_wait` 
instead of removed `wait_event`?

It is a quiet dummy way to confirm that memory corruption had gone, and not 
just moved to somewhere else.

--
wbr, Kirill

> On 2. Nov 2020, at 10:58, Maciej Zdeb  wrote:
> 
> Hi,
> 
> Update for people on the list that might be interested in the issue, because 
> part of discussion was private.
> 
> I wanted to check Willy suggestion and modified h2s struct (added dummy 
> fields):
> 
> struct h2s {
> [...]
> uint16_t status; /* HTTP response status */
> unsigned long long body_len; /* remaining body length according to 
> content-length if H2_SF_DATA_CLEN */
> struct buffer rxbuf; /* receive buffer, always valid (buf_empty or 
> real buffer) */
> int dummy0;
> struct wait_event wait_event; /* Wait list, when we're attempting to 
> send a RST but we can't send */
> int dummy1;
> struct wait_event *recv_wait; /* recv wait_event the conn_stream 
> associated is waiting on (via h2_subscribe) */
> int dummy2;
> struct wait_event *send_wait; /* send wait_event the conn_stream 
> associated is waiting on (via h2_subscribe) */
> int dummy3;
> struct list list; /* To be used when adding in h2c->send_list or 
> h2c->fctl_lsit */
> struct list sending_list; /* To be used when adding in 
> h2c->sending_list */
> };
> 
> With such modified h2s struct, the crash did not occur.
> 
> I've checked HAProxy 2.1, it crashes like 2.0.
> 
> I've also checked 2.2, bisection showed that this commit: 
> http://git.haproxy.org/?p=haproxy-2.2.git;a=commitdiff;h=5723f295d85febf5505f8aef6afabb6b23d6fdec;hp=f11be0ea1e8e571234cb41a2fcdde2cf2161df37
>  
> <http://git.haproxy.org/?p=haproxy-2.2.git;a=commitdiff;h=5723f295d85febf5505f8aef6afabb6b23d6fdec;hp=f11be0ea1e8e571234cb41a2fcdde2cf2161df37>
>  fixed the crashes we experienced. I'm not sure how/if it fixed the memory 
> corruption, it is possible that memory is still corrupted but not causing the 
> crash.
> 
> 
> 
> pt., 25 wrz 2020 o 16:25 Kirill A. Korinsky  <mailto:kir...@korins.ky>> napisał(a):
> Very interesting.
> 
> Anyway, I can see that this pice of code was refactored some time ago: 
> https://github.com/haproxy/haproxy/commit/f96508aae6b49277dcf142caa35042678cf8e2ca
>  
> <https://github.com/haproxy/haproxy/commit/f96508aae6b49277dcf142caa35042678cf8e2ca>
> 
> Maybe it is worth to try 2.2 or 2.3 branch?
> 
> Yes, it is a blind shot and just a guess.
> 
> --
> wbr, Kirill
> 
>> On 25. Sep 2020, at 16:01, Maciej Zdeb > <mailto:mac...@zdeb.pl>> wrote:
>> 
>> Yes at the same place with same value:
>> 
>> (gdb) bt full
>> #0  0x559ce98b964b in h2s_notify_recv (h2s=0x559cef94e7e0) at 
>> src/mux_h2.c:783
>> sw = 0x
>> 
>> 
>> 
>> pt., 25 wrz 2020 o 15:42 Kirill A. Korinsky > <mailto:kir...@korins.ky>> napisał(a):
>> > On 25. Sep 2020, at 15:26, Maciej Zdeb > > <mailto:mac...@zdeb.pl>> wrote:
>> >
>> > I was mailing outside the list with Willy and Christopher but it's worth 
>> > sharing that the problem occurs even with nbthread = 1. I've managed to 
>> > confirm it today.
>> 
>> 
>> I'm curious is it crashed at the same place with the same value?
>> 
>> --
>> wbr, Kirill
>> 
>> 
> 



signature.asc
Description: Message signed with OpenPGP

Re: [2.0.17] crash with coredump

[Kirill A. Korinsky]

Very interesting.

Anyway, I can see that this pice of code was refactored some time ago: 
https://github.com/haproxy/haproxy/commit/f96508aae6b49277dcf142caa35042678cf8e2ca
 
<https://github.com/haproxy/haproxy/commit/f96508aae6b49277dcf142caa35042678cf8e2ca>

Maybe it is worth to try 2.2 or 2.3 branch?

Yes, it is a blind shot and just a guess.

--
wbr, Kirill

> On 25. Sep 2020, at 16:01, Maciej Zdeb  wrote:
> 
> Yes at the same place with same value:
> 
> (gdb) bt full
> #0  0x559ce98b964b in h2s_notify_recv (h2s=0x559cef94e7e0) at 
> src/mux_h2.c:783
> sw = 0x
> 
> 
> 
> pt., 25 wrz 2020 o 15:42 Kirill A. Korinsky  <mailto:kir...@korins.ky>> napisał(a):
> > On 25. Sep 2020, at 15:26, Maciej Zdeb  > <mailto:mac...@zdeb.pl>> wrote:
> >
> > I was mailing outside the list with Willy and Christopher but it's worth 
> > sharing that the problem occurs even with nbthread = 1. I've managed to 
> > confirm it today.
> 
> 
> I'm curious is it crashed at the same place with the same value?
> 
> --
> wbr, Kirill
> 
> 



signature.asc
Description: Message signed with OpenPGP

Re: [2.0.17] crash with coredump

[Kirill A. Korinsky]

> On 25. Sep 2020, at 15:26, Maciej Zdeb  wrote:
> 
> I was mailing outside the list with Willy and Christopher but it's worth 
> sharing that the problem occurs even with nbthread = 1. I've managed to 
> confirm it today.


I'm curious is it crashed at the same place with the same value?

--
wbr, Kirill




signature.asc
Description: Message signed with OpenPGP

Re: [2.0.17] crash with coredump

[Kirill A. Korinsky]

> On 25. Sep 2020, at 15:06, Willy Tarreau  wrote:
> 
> Till here your analysis is right but:
>  - the overflow would only be at most the number of extra threads running
>init_genrand() concurrently, or more precisely the distance between
>the most upfront to the latest thread, so in the worst case nbthread-1
>hence there's no way to write a single location into a totally unrelated
>structure without writing the nbthread-2 words that are between the end
>of the MT array and the overwritten location ;

I don't think so.

We have two threads and steps

1. T1: mti < N
2. T2: mti < N
3. T1:  mtii++
4. T2:  mtii++
5. T1: mt[mti] = (1812433253UL * (mt[mti-1] ^ (mt[mti-1] >> 30)) + mti);
6. T2: mt[mti] = (1812433253UL * (mt[mti-1] ^ (mt[mti-1] >> 30)) + mti);

Let assume that mti = N - 1 at steps 1 and 2. At this case mti (aka MT->o) is N 
+ 1 that is overflow of mt array (MT->v).

So, when T1 writes something at step 5 it puts a random value to mti (aka 
MT->i). Here I assume that the sume has dwrod similar that size(long) and MT 
hasn't got any gaps between v and I.

If so at step 6 we have mti with random value and writes to unpredicted place 
in memory.

>  - the lua code is not threaded (there's a "big lock" around lua since
>stacks cannot easily be protected, though Thierry has some ideas for
>this).

What does not threaded mean here?

I can understand it as all lua function is executed from the only one thread 
(1), or one call is executed at the same thread but different thread can 
execute the same lua code for different TCP/HTTP streams at the same time (2).


--
wbr, Kirill



signature.asc
Description: Message signed with OpenPGP

Re: [2.0.17] crash with coredump

[Kirill A. Korinsky]

Good day,

I'd like to share with your my two cents regarding this topic:

>> lrandom (PRNG for lua, we're using it for 2 or 3 years without any
>> problems, and soon we will drop it from our build)
> 
> Never heard of this last one, not that it would make it suspicious at
> all, just that it might indicate you're having a slightly different
> workload than most common ones and can help spotting directions where
> to look for the problem.


As far as I know Haproxy is using threads by default for some time and I assume 
that Maciej's setup doesn't change anything and it had enabled threads.

If so I believe that lrandom is the root cause of this issue.

I've extracted a pice of code from lrandom and put it here: 
https://gist.github.com/catap/bf862cc0d289083fc1ccd38c905e2416 


You can see that object generator contains N words (and here it is 624), and I 
use an assumption that Maciej's code doesn't create a new generator for each 
request and share lrandom.

Idea of this RNG is initialize each N words via init_genrand and it checking 
that all of them are used, and after one generated a new ones.

Let assume that we called genrand_int32 at the same moment from two threads. If 
condition at lines 39 and 43 are true we start to initialize the next words at 
both threads.

You can see that we can easy move outside of v array at line 21 because two 
threads are increasing i field, and put some random number to i field.

Ans when the second thread is going to line 27 and nobody knows where it put 
0x

Let me quote Willy Tarreau:

> In the trace it's said that sw = 0x. Looking at all places where
> h2s->recv_wait() is modified, it's either NULL or a valid pointer to some
> structure. We could have imagined that for whatever reason h2s is wrong
> here, but this call only happens when its state is still valid, and it
> experiences double dereferences before landing here, which tends to
> indicate that the h2s pointer is OK. Thus the only hypothesis I can have
> for now is memory corruption :-/ That field would get overwritten with
> (int)-1 for whatever reason, maybe a wrong cast somewhere, but it's not
> as if we had many of these.

and base on this I believe that it is the case.

How can it be proved / solved?

I see a few possible options:
1. Switch off threads inside haproxy
2. Use dedicated lrandom per thread
3. Move away from lrandom

As I understand lrandom is using here because it is very fast and secure, and 
reading from /dev/urandom isn't an option.

Here I can suggest to implement Yarrow PRGN (that is very simple to implement) 
with some lua-pure cryptographic hash function.

--
wbr, Kirill



signature.asc
Description: Message signed with OpenPGP

Re: [PATCH] BUILD/SMALL Fixed build on macOS with lua

[Kirill A. Korinsky]

I'd like just confirmed that b684e7 build well.


-- 
wbr, Kirill


> On 5 Mar 2018, at 18:41, Willy Tarreau <w...@1wt.eu> wrote:
> 
> On Mon, Mar 05, 2018 at 02:29:03PM +0000, Kirill A. Korinsky wrote:
>> I missed part of OSX target, my mistake.
> 
> no problem.
> 
>> Meanwhile, it doesn't build with OSX target 
>>> ld: unknown option: -export-dynamic
>> 
>> because at OSX this option calls -export_dynamic
>> 
>> After change option name it works well.
> 
> Ah crap, I know what happened, I had an exchange with Thierry where I
> suggested a change and I very well suspect that he didn't rebuild after
> modifying the patch since his commit message mentions the underscore as
> well. I'm fixing it and pushing it again :-)
> 
> Thanks for the test!
> 
> Willy

Re: [PATCH] BUILD/SMALL Fixed build on macOS with lua

[Kirill A. Korinsky]

> On 5 Mar 2018, at 18:24, Willy Tarreau <w...@1wt.eu> wrote:
> 
> On Mon, Mar 05, 2018 at 02:10:34PM +0000, Kirill A. Korinsky wrote:
>> Hi Will,
>> 
>> I used a last master 13113d6abb and command to build:
>> 
>>> make USE_LUA=1 LUA_LIB=/usr/local/opt/lua/lib LUA_LIB_NAME=lua.5.3 
>>> TARGET=generic
> 
> The flag was set for the "osx" target only, not for "generic".
> By the way you shouldn't use "generic", it will ruin your performance.
> For example your OS supports kqueue() while generic supports only poll().

I missed part of OSX target, my mistake.

Meanwhile, it doesn't build with OSX target 
> ld: unknown option: -export-dynamic

because at OSX this option calls -export_dynamic

After change option name it works well.

-- 
wbr, Kirill

Re: [PATCH] BUILD/SMALL Fixed build on macOS with lua

[Kirill A. Korinsky]

Hi Will,

I used a last master 13113d6abb and command to build:

> make USE_LUA=1 LUA_LIB=/usr/local/opt/lua/lib LUA_LIB_NAME=lua.5.3 
> TARGET=generic


anyway, it hasn't build with same issue.

> gcc  -g -o haproxy src/ev_poll.o src/hlua.o src/hlua_fcn.o ebtree/ebtree.o 
> ebtree/eb32sctree.o ebtree/eb32tree.o ebtree/eb64tree.o ebtree/ebmbtree.o 
> ebtree/ebsttree.o ebtree/ebimtree.o ebtree/ebistree.o src/proto_http.o 
> src/cfgparse.o src/server.o src/stream.o src/flt_spoe.o src/stick_table.o 
> src/stats.o src/mux_h2.o src/checks.o src/haproxy.o src/log.o src/dns.o 
> src/peers.o src/standard.o src/sample.o src/cli.o src/stream_interface.o 
> src/proto_tcp.o src/backend.o src/proxy.o src/tcp_rules.o src/listener.o 
> src/flt_http_comp.o src/pattern.o src/cache.o src/filters.o src/vars.o 
> src/acl.o src/payload.o src/connection.o src/raw_sock.o src/proto_uxst.o 
> src/flt_trace.o src/session.o src/ev_select.o src/channel.o src/task.o 
> src/queue.o src/applet.o src/map.o src/frontend.o src/freq_ctr.o 
> src/lb_fwlc.o src/mux_pt.o src/auth.o src/fd.o src/hpack-dec.o src/memory.o 
> src/lb_fwrr.o src/lb_chash.o src/lb_fas.o src/hathreads.o src/chunk.o 
> src/lb_map.o src/xxhash.o src/regex.o src/shctx.o src/buffer.o src/action.o 
> src/h1.o src/compression.o src/pipe.o src/namespace.o src/sha1.o 
> src/hpack-tbl.o src/hpack-enc.o src/uri_auth.o src/time.o src/proto_udp.o 
> src/arg.o src/signal.o src/protocol.o src/lru.o src/hdr_idx.o 
> src/hpack-huff.o src/mailers.o src/h2.o src/base64.o src/hash.o   
> -Wl,--export-dynamic -L/usr/local/opt/lua/lib -llua.5.3 -lm
> ld: unknown option: --export-dynamic
> clang: error: linker command failed with exit code 1 (use -v to see 
> invocation)
> make: *** [haproxy] Error 1


I also attached whole log of building to this email.



if it fine, I can send updated (based on last master) version of my patch with 
uname ASAP.

--
wbr, Kirill


> On 5 Mar 2018, at 18:00, Willy Tarreau <w...@1wt.eu> wrote:
>
> Hi Kirill,
>
> On Sun, Mar 04, 2018 at 01:15:15PM +0400, Kirill A. Korinsky wrote:
>> Last macOS has renamed flag --export-dynamic to -export_dynamic, just fixed 
>> it
>
> Thanks for this. In parallel Thierry faced the same issue and sent me a
> slightly different one that I preferred and merged (it doesn't involve
> uname which is for the host OS instead of the target). It uses a new
> EXPORT_SYMBOL directive which indicates the option name, and which is
> set only for the OSX target, so that it can easily be set for any other
> target if needed.
>
> Please check the latest master to confirm it's fine for you as well.
>
> Thanks,
> Willy



log.gz
Description: log.gz

[PATCH] BUILD/SMALL Fixed build on macOS with lua

[Kirill A. Korinsky]

Last macOS has renamed flag --export-dynamic to -export_dynamic, just fixed it
---
 Makefile | 4 
 1 file changed, 4 insertions(+)

diff --git a/Makefile b/Makefile
index 2acf5028..914e3d19 100644
--- a/Makefile
+++ b/Makefile
@@ -630,7 +630,11 @@ check_lua_inc = $(shell if [ -d $(2)$(1) ]; then echo 
$(2)$(1); fi;)
 
 BUILD_OPTIONS   += $(call ignore_implicit,USE_LUA)
 OPTIONS_CFLAGS  += -DUSE_LUA $(if $(LUA_INC),-I$(LUA_INC))
+ifeq ($(shell uname), Darwin)
+LUA_LD_FLAGS := -Wl,-export_dynamic $(if $(LUA_LIB),-L$(LUA_LIB))
+else
 LUA_LD_FLAGS := -Wl,--export-dynamic $(if $(LUA_LIB),-L$(LUA_LIB))
+endif
 ifeq ($(LUA_LIB_NAME),)
 # Try to automatically detect the Lua library
 LUA_LIB_NAME := $(firstword $(foreach lib,lua5.3 lua53 lua,$(call 
check_lua_lib,$(lib),$(LUA_LD_FLAGS
-- 
2.16.1

Re: [PATCH] BUILD/SMALL Fixed build on macOS with lua

[Kirill A. Korinsky]

Hey,

Sorry for long response.

Your way much better.

I will prepare a patch ASAP.

-- 
wbr, Kirill


> On 4 Jan 2018, at 19:24, Thierry Fournier <tfourn...@arpalert.org> wrote:
> 
> 
>> On 4 Jan 2018, at 15:16, Kirill A. Korinsky <kir...@korins.ky> wrote:
>> 
>> Honestly, I didn't.
>> 
>> If I right understand how export-dynamic works and how haproxy use 
>> integrated LUA, it shouldn't have any impact.
>> 
>> Honestly I see only one case when export-dynamic requests: when some 
>> application load haproxy over dlopen, and use some function from haproxy 
>> binary object.
>> 
>> I expect that it isn't true, is it?
> 
> 
> 
> Hi Kirill,
> 
> This option is usefull for load Lua extensions as .so files. Something
> like the openSSL Lua bindings is provided as .so Lua module. These kind
> of modules requires some Lua symbol which are not used by HAProxy, so
> without the option “ --export-dynamic” the load of these libraries fail
> with a message explaining that some symbols are missing.
> 
> Note: I’m not specialised in the compilation options, and maybe something
> following is wrong.
> 
> The flag --export-dynamic force all the symbols (of the Lua library) to
> be exported in the ELF binary. I guess that this option force the linker
> to embed also the unused symbols from the Lua library.
> 
> The man on my mac compiler shows the option “-export_dynamic” (with
> underscore in place of dash). Maybe a solution is to detect the platform
> and set the right option.
> 
> br,
> Thierry
> 
>> 
>> -- 
>> wbr, Kirill
>> 
>> 
>>> On 4 Jan 2018, at 01:10, Willy Tarreau <w...@1wt.eu> wrote:
>>> 
>>> Hi Kirill,
>>> 
>>> On Thu, Dec 28, 2017 at 04:13:38AM +0400, Kirill A. Korinsky wrote:
>>>> Last macOS hasn't support export-dynamic, just remove it
>>>> ---
>>>> Makefile | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>> 
>>>> diff --git a/Makefile b/Makefile
>>>> index 2acf5028..19234897 100644
>>>> --- a/Makefile
>>>> +++ b/Makefile
>>>> @@ -630,7 +630,7 @@ check_lua_inc = $(shell if [ -d $(2)$(1) ]; then echo 
>>>> $(2)$(1); fi;)
>>>> 
>>>> BUILD_OPTIONS   += $(call ignore_implicit,USE_LUA)
>>>> OPTIONS_CFLAGS  += -DUSE_LUA $(if $(LUA_INC),-I$(LUA_INC))
>>>> -LUA_LD_FLAGS := -Wl,--export-dynamic $(if $(LUA_LIB),-L$(LUA_LIB))
>>>> +LUA_LD_FLAGS := $(if $(LUA_LIB),-L$(LUA_LIB))
>>> 
>>> Hmmm how can you be sure you didn't break anything else ? I'm pretty
>>> sure that there was a reason for adding this --export-dynamic, maybe
>>> certain things will still not work on your platform, or others won't
>>> work at all. We need to run some checks before taking this one.
>>> 
>>> I'm CCing Thierry in case he reminds why we need this.
>>> 
>>> Regards,
>>> Willy
>> 
> 

Re: [PATCH] BUILD/SMALL Fixed build on macOS with lua

[Kirill A. Korinsky]

Honestly, I didn't.

If I right understand how export-dynamic works and how haproxy use integrated 
LUA, it shouldn't have any impact.

Honestly I see only one case when export-dynamic requests: when some 
application load haproxy over dlopen, and use some function from haproxy binary 
object.

I expect that it isn't true, is it?

-- 
wbr, Kirill


> On 4 Jan 2018, at 01:10, Willy Tarreau <w...@1wt.eu> wrote:
> 
> Hi Kirill,
> 
> On Thu, Dec 28, 2017 at 04:13:38AM +0400, Kirill A. Korinsky wrote:
>> Last macOS hasn't support export-dynamic, just remove it
>> ---
>> Makefile | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/Makefile b/Makefile
>> index 2acf5028..19234897 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -630,7 +630,7 @@ check_lua_inc = $(shell if [ -d $(2)$(1) ]; then echo 
>> $(2)$(1); fi;)
>> 
>> BUILD_OPTIONS   += $(call ignore_implicit,USE_LUA)
>> OPTIONS_CFLAGS  += -DUSE_LUA $(if $(LUA_INC),-I$(LUA_INC))
>> -LUA_LD_FLAGS := -Wl,--export-dynamic $(if $(LUA_LIB),-L$(LUA_LIB))
>> +LUA_LD_FLAGS := $(if $(LUA_LIB),-L$(LUA_LIB))
> 
> Hmmm how can you be sure you didn't break anything else ? I'm pretty
> sure that there was a reason for adding this --export-dynamic, maybe
> certain things will still not work on your platform, or others won't
> work at all. We need to run some checks before taking this one.
> 
> I'm CCing Thierry in case he reminds why we need this.
> 
> Regards,
> Willy

[PATCH] BUILD/SMALL Fixed build on macOS with lua

[Kirill A. Korinsky]

Last macOS hasn't support export-dynamic, just remove it
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 2acf5028..19234897 100644
--- a/Makefile
+++ b/Makefile
@@ -630,7 +630,7 @@ check_lua_inc = $(shell if [ -d $(2)$(1) ]; then echo 
$(2)$(1); fi;)
 
 BUILD_OPTIONS   += $(call ignore_implicit,USE_LUA)
 OPTIONS_CFLAGS  += -DUSE_LUA $(if $(LUA_INC),-I$(LUA_INC))
-LUA_LD_FLAGS := -Wl,--export-dynamic $(if $(LUA_LIB),-L$(LUA_LIB))
+LUA_LD_FLAGS := $(if $(LUA_LIB),-L$(LUA_LIB))
 ifeq ($(LUA_LIB_NAME),)
 # Try to automatically detect the Lua library
 LUA_LIB_NAME := $(firstword $(foreach lib,lua5.3 lua53 lua,$(call 
check_lua_lib,$(lib),$(LUA_LD_FLAGS
-- 
2.15.1