Re: lua haproxy-auth-request - round 2

[Tim Düsterhus]

Bob,

Am 05.10.2018 um 22:10 schrieb Computerisms Corporation:
> Okay, this is something I hadn't looked at.  Not 100% sure I am
> interpreting correctly, but assuming I am, then the browser is reporting
> that the connection is successfully upgrading to websockets both with
> and without the Lua script enabled.  As far as I can tell, the only
> thing that changes is the cookies and keys.  There is a significant
> difference in the "waiting" response, though; ~350ms with the lua
> script, but less than 5ms without it.

Use your browser's dev tools (F12 for Firefox, Chrome). Search for the
websocket request. It's just called '/websocket' for Etherpad. It should
return a 101 Switching Protocols. In Google Chrome you can even look at
the WebSocket Frames.

>> b) Send credentials for basic authentication for Websockets.
> 
> hm, so I don't seem to be able to connect to etherpad directly using a
> ws:// schematic in chrome or firefox, and I think that is what you mean.
>  not sure if that is just me, though, will work some more on that and
> see if I can figure out if I am doing something wrong there...

No, use the dev tools and look at the request headers, whether there is
an Authorization header. Of course oauth_proxy needs to be enabled.

I attached a screenshot of Chrome's dev tools.

If you send me credentials in private I can take a look myself, if you want.

Best regards
Tim Düsterhus

Re: haproxy support for stratum protocol ?

[Willy Tarreau]

On Tue, Oct 09, 2018 at 04:58:37PM +, jdelo...@protonmail.com wrote:
> > > My use case is to try to set up a mining farm based on Raspberry-Pis. I 
> > > did a
> > > study showing that it's much more efficient for the same price or power 
> > > user
> > > as a PC :
> > > https://www.linkedin.com/pulse/more-efficient-mining-raspberry-pi-julien-delorme/
> >
> > Fun project there reminding me the build farm. I've left a comment with
> > some links to much better hardware than RPis :-)
> 
> Oh thanks, I will check there!

No, thanks to you, I've just found in your patch here an example of how
to use ARMv8's CRC32 instruction (which is trivial by the way) and which
has been staying on my todo list for ages :

   https://github.com/tpruvot/cpuminer-multi/pull/29

We use CRC32 in libslz for the gzip compression, I should give it a try,
it could bring gzip performance closer to deflate on ARM64, which could
be nice for those running hosting services on cheap ARM servers.

> > > And since haproxy is already used a lot and recommanded for this use case
> > > (principally to give active-backup fonction between pools), I think it 
> > > will
> > > interest many of users to be able to improve their capacity and reduce the
> > > risk to be banned.
> >
> > I was not aware of this. Do you have any link ?
> 
> My pleasure!
>   
> https://www.reddit.com/r/Monero/comments/5io4u3/simple_mining_proxy_setup_using_haproxy_or_who/
>   https://github.com/xmrig/xmrig-proxy/wiki/TLS
>   https://imgur.com/a/wrPw6
>   https://github.com/turtlecoin/turtlecoin/wiki/TurtleCoind-behind-HAProxy
> 
> As you can see its pretty common. People use it for automatic fail-over to
> cover maintenance windows, so its placed on the client side, which is not
> very common I think.

I see, that probably makes sense, indeed. We're used to say that haproxy
is a Swiss-army-knife anyway :-)

Willy

Re: haproxy support for stratum protocol ?

[jdelorme]

> > My use case is to try to set up a mining farm based on Raspberry-Pis. I did 
> > a
> > study showing that it's much more efficient for the same price or power user
> > as a PC :
> > https://www.linkedin.com/pulse/more-efficient-mining-raspberry-pi-julien-delorme/
>
> Fun project there reminding me the build farm. I've left a comment with
> some links to much better hardware than RPis :-)

Oh thanks, I will check there!

> > And since haproxy is already used a lot and recommanded for this use case
> > (principally to give active-backup fonction between pools), I think it will
> > interest many of users to be able to improve their capacity and reduce the
> > risk to be banned.
>
> I was not aware of this. Do you have any link ?

My pleasure!
  
https://www.reddit.com/r/Monero/comments/5io4u3/simple_mining_proxy_setup_using_haproxy_or_who/
  https://github.com/xmrig/xmrig-proxy/wiki/TLS
  https://imgur.com/a/wrPw6
  https://github.com/turtlecoin/turtlecoin/wiki/TurtleCoind-behind-HAProxy

As you can see its pretty common. People use it for automatic fail-over to 
cover maintenance windows, so its placed on the client side, which is not very 
common I think.

> However the protocol looks easy enough to parse (JSON?) so I wouldn't be
> surprised if you could implement it entirely using Lua.

I don't have much knowledge there so I'll possibly have to wait for someone 
else to have this idea ^^

Thanks,
J.

Re: haproxy support for stratum protocol ?

[Willy Tarreau]

On Tue, Oct 09, 2018 at 04:15:10PM +, jdelo...@protonmail.com wrote:
> Yes, that's it. I was not aware of this spec.
> 
> My use case is to try to set up a mining farm based on Raspberry-Pis. I did a
> study showing that it's much more efficient for the same price or power user
> as a PC :
> 
> https://www.linkedin.com/pulse/more-efficient-mining-raspberry-pi-julien-delorme/

Fun project there reminding me the build farm. I've left a comment with
some links to much better hardware than RPis :-)

> And since haproxy is already used a lot and recommanded for this use case
> (principally to give active-backup fonction between pools), I think it will
> interest many of users to be able to improve their capacity and reduce the
> risk to be banned.

I was not aware of this. Do you have any link ?

> So do you think it is something possible ?

Well, a quick look at the spec suggests HTTP for old versions and TCP for
new ones if I understand correctly. HTTP could theorically be merged using
"http-reuse" and "maxconn 1" on the server. However you will not get any
pipelining, I don't know if that's important or not (if the server takes
a long time to respond, your clients will have to wait for other responses
first). For the TCP one, I'm pretty certain that for now it's not supported,
and I don't really have an idea about what would be needed to support this,
very likely implementing multiplexed transactions, which would be more or
less as complicated as HTTP pipelining.

However the protocol looks easy enough to parse (JSON?) so I wouldn't be
surprised if you could implement it entirely using Lua.

Regards,
Willy

Re: haproxy support for stratum protocol ?

[jdelorme]

Yes, that's it. I was not aware of this spec.

My use case is to try to set up a mining farm based on Raspberry-Pis. I did a 
study showing that it's much more efficient for the same price or power user as 
a PC :

https://www.linkedin.com/pulse/more-efficient-mining-raspberry-pi-julien-delorme/

The problem is that some pools will ban my IP address if I create too many 
parallel connections, so it makes sense to use an agregating proxy to do this :

 =>
  30*RPi => haproxy => pool (1 single connection)
 =>

And since haproxy is already used a lot and recommanded for this use case 
(principally to give active-backup fonction between pools), I think it will 
interest many of users to be able to improve their capacity and reduce the risk 
to be banned.

So do you think it is something possible ?

J.

‐‐‐ Original Message ‐‐‐
On Tuesday, October 9, 2018 4:06 PM, Aleksandar Lazic  
wrote:

> Hi.
>
> Am 09.10.2018 um 15:04 schrieb jdelo...@protonmail.com:
>
> > Hello,
> > Please CC me as I'm not subscribed on the list.
> > Does haproxy will consider support for the stratum protocol ?
>
> Do you mean this?
> http://stratumprotocol.org/
>
> > Thanks,
> > Julien
>
> Regards
> Aleks

Re: haproxy support for stratum protocol ?

[Aleksandar Lazic]

Hi.

Am 09.10.2018 um 15:04 schrieb jdelo...@protonmail.com:
> Hello,
> 
> Please CC me as I'm not subscribed on the list.
> Does haproxy will consider support for the stratum protocol ?

Do you mean this?
http://stratumprotocol.org/

> Thanks,
> Julien

Regards
Aleks

Re: Fix some warnings and a small bug in debug logic

[Willy Tarreau]

Hi guys,

On Sun, Oct 07, 2018 at 02:20:30PM +0200, Lukas Tribus wrote:
> > Is there any interest in these patches?
> 
> I'm sure there is. Looping in Willy.

Yep, thanks, I've just merged them now.

Cheers,
Willy

haproxy support for stratum protocol ?

[jdelorme]

Hello,

Please CC me as I'm not subscribed on the list.
Does haproxy will consider support for the stratum protocol ?

Thanks,
Julien

Re: haproxy bug: healthcheck not passing after port change when statefile is enabled

[Sven Wiltink]

Hey Baptiste,


We noticed the SRV patch has been merged. That should mean that we can now fix 
this issue as well. Would you be able to fix this or should we

try to provide a patch?


Thanks again in advance,

Sven


Van: Baptiste 
Verzonden: donderdag 12 juli 2018 14:52:24
Aan: Sven Wiltink
CC: haproxy@formilux.org
Onderwerp: Re: haproxy bug: healthcheck not passing after port change when 
statefile is enabled

Hi Sven,

Thanks for the clarification.
It's a bit more complicated than what it is supposed to be.
I think we may want to apply the port only if it has been changed at runtime 
(changed by DNS SRV records).

The status is the following: I have a pending patch which brings SRV record 
information into the state file. (WIP, but last mile)
Once it has been merged, we'll be able to fix this issue (by applying the port 
only when the server is being managed by an SRV record).

Baptiste


On Tue, Jul 3, 2018 at 3:41 PM, Sven Wiltink 
mailto:swilt...@transip.nl>> wrote:

Hey Baptiste,


Thank you for looking into it.


The bug is triggered by running haproxy with the following config:


global
maxconn 32000
tune.maxrewrite 2048
user haproxy
group haproxy
daemon
chroot /var/lib/haproxy
nbproc 1
maxcompcpuusage 85
spread-checks 0
stats socket /var/run/haproxy.sock mode 600 level admin process 1 user 
haproxy group haproxy
server-state-file test
server-state-base /var/run/haproxy/state
master-worker no-exit-on-failure

defaults
load-server-state-from-file global
log global
timeout http-request 5s
timeout connect  2s
timeout client   300s
timeout server   300s
mode http
option dontlog-normal
option http-server-close
option redispatch
option log-health-checks

listen stats
bind :1936
bind-process 1
mode http
stats enable
stats uri /
stats admin if TRUE

listen banaan-443-ipv4
bind :443
mode tcp
server banaan-vps 127.0.0.1:443 check inter 2000


- Then start haproxy (it will do healthchecks to port 443)
- change server banaan-vps 127.0.0.1:443 check inter 2000 
to server banaan-vps 127.0.0.1:80 check inter 2000
- save the state using /bin/sh -c "echo show servers state | /usr/bin/socat 
/var/run/haproxy.sock - > /var/run/haproxy/state/test" (this is normally done 
using the systemd file on reload, see initial mail)
- reload haproxy (it still does healthchecks to port 443 while port 80 was 
expected)

if you delete the statefile and reload haproxy it will start healthchecks for 
port 80 as expected

-Sven








Van: Baptiste mailto:bed...@gmail.com>>
Verzonden: dinsdag 3 juli 2018 11:38:14
Aan: Sven Wiltink
CC: haproxy@formilux.org
Onderwerp: Re: haproxy bug: healthcheck not passing after port change when 
statefile is enabled

Hi Sven,

Thanks a lot for your feedback!
I'll check how we could handle this use case with the state file.

Just to ensure I'm going to troubleshoot the right issue, could you please 
summarize how you trigger this issue in a few simple steps?
IE:
- conf v1, server port is X
- generate server state (where port is X)
- update conf to v2, where port is Y
reload HAProxy => X is applied, while you expect to get Y instead

Baptiste



On Mon, Jun 25, 2018 at 12:55 PM, Sven Wiltink 
mailto:swilt...@transip.nl>> wrote:

Hello,


So we've dug a little deeper and the issue seems to be caused by the port value 
in the statefile. When the target port of a server has changed between reloads 
the port specified in the state file is leading. When running tcpdump you can 
see the healthchecks are being performed for the old port. After stopping 
haproxy and removing the statefile the healthcheck is performed for the right 
port. When manually editing the statefile to a random port the healthchecks 
will be performed for that port instead of the one specified by the config.


The code responsible for this is line 
http://git.haproxy.org/?p=haproxy-1.8.git;a=blob;f=src/server.c;h=523289e3bda7ca6aa15575f1928f5298760cf582;hb=HEAD#l2931

from commit 
http://git.haproxy.org/?p=haproxy-1.8.git;a=commitdiff;h=3169471964fdc49963e63f68c1fd88686821a0c4.


A solution would be invalidating the state when the ports don't match.


-Sven




Van: Sven Wiltink
Verzonden: dinsdag 12 juni 2018 17:01:18
Aan: haproxy@formilux.org
Onderwerp: haproxy bug: healthcheck not passing after port change when 
statefile is enabled

Hello,

There seems to be a bug in the loading of state files after a configuration 
change. When changing the destination port of a server the healthchecks never 
start passing if the state before the reload was down. This bug has been 
introduced after 1.7.9 as we cannot reproduce it on machines running that 
version of haproxy. You can use 

Re: faster than load-server-state-from-file?

[Baptiste]

On Mon, Oct 8, 2018 at 7:57 PM Aleksandar Lazic  wrote:

> Am 08.10.2018 um 19:35 schrieb Willy Tarreau:
> > On Mon, Oct 08, 2018 at 07:27:39PM +0200, Aleksandar Lazic wrote:
> >> Hi Baptiste.
> >>
> >> Am 08.10.2018 um 16:20 schrieb Baptiste:
> >>> Bonjour Messieurs,
> >>>
> >>> (je passe en FR et hors ML et je top-poste!!!).
> >>
> >> Just for my curiosity, why not answering in english?
> >
> > He thought he responded privately and excluded the mailing list from
> > the CC but apparently he was facing an ENOCOFFEE type of error :-)
>
> Oh yes we all know this error code ;-)
>
> > Cheers,
> > Willy
>
> Regards
> Aleks
>

That's it, furthermore, Willy, Pierre and I speaks the same protocol (the
French Language)...
I "switched" to private, cause I'm asking some "internal" information to be
able to reproduce the behavior and to troubleshoot it.

Sorry for the noise, beers, tea or coffee are on me!

Baptiste