Re: [OPEN-ILS-GENERAL] Web Client and Load Balancer

2019-09-03 Thread Driscoll, Martha
Hi Josh,
I did not get pound to work with websockets.  I switched to ldirectord.  I
did this setup a year ago so I don't know if the fix in January resolved
the problem.  I have not tested it.  I'm also using apache-websockets and
have not yet switched to websocketd.  Not sure if that make a difference
with pound or not.

Martha Driscoll, Systems Manager
North of Boston Library Exchange
www.noblenet.org




On Tue, Sep 3, 2019 at 9:29 AM Josh Stompro 
wrote:

> Martha, I just saw your post.  We are also using pound proxy, and looking
> to upgrade to 3.3 and use the web staff client.  Did you get it to work
> with pound.
>
> It looks like historically pound proxy has had trouble with websockets
> [1], but there was a release early this year with a fix perhaps [2]?
>
> What is everyone switching to from pound?  HAproxy?
>
> Josh
>
> 1 -
> https://superuser.com/questions/542204/pound-configure-to-handle-websocket
> 2. -
> http://www.apsis.ch/pound/pound_list/archive/2019/2019-01/1547402746000#1547402746000
>
> On Mon, Aug 6, 2018 at 8:41 AM Martha Driscoll 
> wrote:
>
>> We are running release 3.0.9 and I can connect to the web client if I
>> browse directly to one of my application servers.  When I try to login via
>> the load balancer I get the login screen but when I try to login it just
>> refreshes the login screen.
>>
>> Is there something that needs to be configured for the web client to work
>> through a load balancer?  We are using the pound load balancer and I have
>> backends setup for port 443.
>>
>> Martha Driscoll, Systems Manager
>> North of Boston Library Exchange
>> www.noblenet.org
>>
>>
>>
>
> --
> Josh Stompro - IT Director
> Lake Agassiz Regional Library
> Desk: 218-233-3757 Ext 139
> Cell: 218-790-2110
>


Re: [OPEN-ILS-GENERAL] Web Client and Load Balancer

2019-09-03 Thread Bill Erickson
On Tue, Sep 3, 2019 at 10:52 AM Jason Stephenson  wrote:

> On 9/3/19 10:11 AM, Bill Erickson wrote:
>
> > After some poking around I made 2 changes
> >
> > 1. Use an IP address (e.g. 127.0.0.1) in the proxy_pass configuration
> > instead of a hostname to bypass DNS.
> > 2. Drop the keepalive timeout from the default of 65s to 5s or so.
>
> There is no keepalive timeout in the nginx proxy configuration, at least
> not the default/sample config.  Are you talking about a different
> setting, or does it need to be added?
>

Sorry, Jason.  See keepalive_timeout in /etc/nginx/nginx.conf.  I have not
yet looked to see if I can override the value in the site config.

-b


Re: [OPEN-ILS-GENERAL] Web Client and Load Balancer

2019-09-03 Thread Jason Stephenson
On 9/3/19 10:11 AM, Bill Erickson wrote:

> After some poking around I made 2 changes
> 
> 1. Use an IP address (e.g. 127.0.0.1) in the proxy_pass configuration
> instead of a hostname to bypass DNS.  
> 2. Drop the keepalive timeout from the default of 65s to 5s or so.

There is no keepalive timeout in the nginx proxy configuration, at least
not the default/sample config.  Are you talking about a different
setting, or does it need to be added?




Re: [OPEN-ILS-GENERAL] Web Client and Load Balancer

2019-09-03 Thread Bill Erickson
Hi,

In my initial testing of NGINX with and without a load balancer, one thing
I ran into were intermittent page load delays.  The browser would briefly
flash a "cannot load page" message (I don't recall the exact wording), then
quickly refresh and and the page would load fine.  Similarly, monitoring
tools would occasionally think a host was down, only to report soon after
the host was fine.

After some poking around I made 2 changes

1. Use an IP address (e.g. 127.0.0.1) in the proxy_pass configuration
instead of a hostname to bypass DNS.
2. Drop the keepalive timeout from the default of 65s to 5s or so.

#1 was mostly a Hail Mary.  I'm pretty sure #2 had the bigger impact.

#2 is also important if you use the load balancer to support "detaching"
bricks for zero-downtime updates.  If the keepalive is too long, it takes a
long time for clients to disconnect after taking a brick out of rotation.

I could also imagine long keepalives causing the load balancer to keep more
idle connections open, thus using more memory.

-b








On Tue, Sep 3, 2019 at 9:52 AM Jason Stephenson  wrote:

> Josh,
>
> It's interesting to me that you reply to Martha's email today.
>
> We recently started using nginx proxy on the brick heads with ldirectord
> running on our load balancers.  We've had issues where ldirectord dies
> about one per week on each load balancer.  It appears that the load
> balancer is running out of memory.  I've mentioned this in IRC a couple
> of times but gotten no responses from anyone.
>
> I am going to look at alternatives, and I'm leaning toward setting up
> haproxy on the load balancers.
>
> Jason
>
>


Re: [OPEN-ILS-GENERAL] Web Client and Load Balancer

2019-09-03 Thread Jason Stephenson
Josh,

It's interesting to me that you reply to Martha's email today.

We recently started using nginx proxy on the brick heads with ldirectord
running on our load balancers.  We've had issues where ldirectord dies
about one per week on each load balancer.  It appears that the load
balancer is running out of memory.  I've mentioned this in IRC a couple
of times but gotten no responses from anyone.

I am going to look at alternatives, and I'm leaning toward setting up
haproxy on the load balancers.

Jason



Re: [OPEN-ILS-GENERAL] Web Client and Load Balancer

2019-09-03 Thread Josh Stompro
Martha, I just saw your post.  We are also using pound proxy, and looking
to upgrade to 3.3 and use the web staff client.  Did you get it to work
with pound.

It looks like historically pound proxy has had trouble with websockets [1],
but there was a release early this year with a fix perhaps [2]?

What is everyone switching to from pound?  HAproxy?

Josh

1 -
https://superuser.com/questions/542204/pound-configure-to-handle-websocket
2. -
http://www.apsis.ch/pound/pound_list/archive/2019/2019-01/1547402746000#1547402746000

On Mon, Aug 6, 2018 at 8:41 AM Martha Driscoll 
wrote:

> We are running release 3.0.9 and I can connect to the web client if I
> browse directly to one of my application servers.  When I try to login via
> the load balancer I get the login screen but when I try to login it just
> refreshes the login screen.
>
> Is there something that needs to be configured for the web client to work
> through a load balancer?  We are using the pound load balancer and I have
> backends setup for port 443.
>
> Martha Driscoll, Systems Manager
> North of Boston Library Exchange
> www.noblenet.org
>
>
>

-- 
Josh Stompro - IT Director
Lake Agassiz Regional Library
Desk: 218-233-3757 Ext 139
Cell: 218-790-2110