William, HAProxy says it was trying to establish a connection to the server. (I have not yet take a look at the pcap files)
Please add some timeouts in your defaults section: timeout connect 3s timeout server 30s timeout client 30s and run again the test, and let us know the logs generated by HAProxy. Baptiste On Tue, Sep 30, 2014 at 2:14 PM, William Lewis <[email protected]> wrote: > Hi Baptiste / Benjamin, > > I've attached haproxy log, and 3 pcap files > > The test with haproxy ended with me killing the node process, so the event > source request terminated and the hanging resource requests 503'd as shown at > the end of the log. > > Looking at the tcpdumps. > > 1. With haproxy > > * You can see that there are 6 concurrent http connections between iOS and > haproxy. > * In the first connection stream you can see the initial document, followed > by the event stream > * Then you can see the client has used http piping (pretty dumb considering > the browser should know this connection is occupied) to send requests for > /21.png /22.png /23.png ( the hanging resources) > * The first connection stream carried on responding with data from the event > source and the stuck resources are eventually 503'd when the node app is > killed > > 2. Without haproxy > > * This time we there are 12 distinct http connections that have been made > between iOS and node > * Again in the first connection stream you see the initial document followed > by the event stream and the pipelined requests for the same resources that go > stuck above > * However this time after the next event is emitted by the event stream, the > connection is terminated and carries on with a new connection > * And you see this in the browser console, but the event stream carries on > seamlessly > > > > > * The requests that were piped lined in that request get dealt with in other > streams e.g. /21.png is in stream 8 > > > I am by no means an expert analysing tcpdumps or how http pipelining is > supposed to work but it looks to me that without haproxy in the middle node > has managed to identify there are requests stuck in a http pipeline and reset > the connection to allow the browser to continue. Is there anyway to achieve > the same with haproxy? > > > > > > > > > > On 30 Sep 2014, at 12:21, Benjamin Lee <[email protected]> wrote: > >> On 30 September 2014 18:54, Baptiste <[email protected]> wrote: >>> >>> >>> >>> On Mon, Sep 29, 2014 at 9:15 PM, William Lewis <[email protected]> wrote: >>>> >>>> Hi all >>>> >>>> I have a problem with a website which uses Server-Sent Events where the >>>> long lived connection for the Server Events seems to be blocking other >>>> resources from loading on iOS clients only and only when I have haproxy >>>> between client and server. >>>> >>>> This is my test case. >>>> >>>> * Create a node express app which serves a html page which subscribes to >>>> an EventSource and asynchronously adds 200 300x100px images to the DOM >>>> * Node app is configured to serve resources with 500ms delay to reliably >>>> reproduce the problem >>>> * Configure basic haproxy between node app and client >>>> * Reset cache on iOS device and connect to server >>>> >>>> Expected result >>>> >>>> * Client open 5 simultaneous http connections to the server >>>> * 1 connection is blocked listening for events from the EventSource >>>> * The remaining 4 connections are used to download the 200 images >>>> >>>> Actual Result >>>> >>>> * Connection to EventSource is established and events start to be logged >>>> to the console >>>> * Images start to download on the page >>>> * Several of the images get blocked and never load >>>> >>>> >>>> Clearing the device cache and connecting directly to the server, all >>>> resources load, although the loading pattern of images is significantly >>>> different. >>>> >>>> If anyone has any ideas I would greatly appreciate any suggestions?? >>>> >>>> >>>> Sources and config included below. >>>> >>>> * index.html >>>> >>>> <html> >>>> <head> >>>> <style> >>>> img { >>>> width: 30px; >>>> height: 10px; >>>> border-style: solid; >>>> border-color: black; >>>> border-width: 1px; >>>> } >>>> </style> >>>> </head> >>>> <body> >>>> <script type="text/javascript"> >>>> var source = new EventSource('/events'); >>>> >>>> source.onmessage = function(e) { >>>> console.log(e.data); >>>> } >>>> >>>> var body = document.querySelectorAll('body'); >>>> var createImage = function(i) { >>>> var element = document.createElement('img'); >>>> element.src = '/' + i + '.png'; >>>> >>>> body[0].appendChild(element); >>>> } >>>> >>>> window.setTimeout(function() { >>>> for (var i = 1; i < 200; i++) { >>>> createImage(i); >>>> } >>>> }, 1000); >>>> </script> >>>> </body> >>>> </html> >>>> >>>> * app.js >>>> >>>> var express = require('express'); >>>> var app = express(); >>>> >>>> app.get('/events', function(req, res) { >>>> >>>> // let request last as long as possible >>>> req.socket.setTimeout(Infinity); >>>> >>>> var messageCount = 0; >>>> >>>> res.writeHead(200, { >>>> 'Content-Type': 'text/event-stream', >>>> 'Cache-Control': 'no-cache', >>>> 'Connection': 'keep-alive' >>>> }); >>>> res.write('\n'); >>>> >>>> var timeout; >>>> >>>> var emitEvent = function() { >>>> res.write('id:' + ++messageCount + '\n'); >>>> res.write('data:' + new Date().getTime() + '\n\n'); >>>> >>>> timeout = setTimeout(emitEvent, 3000); >>>> } >>>> >>>> req.on("close", function() { >>>> clearTimeout(timeout); >>>> }); >>>> >>>> emitEvent(); >>>> >>>> }); >>>> >>>> var staticHandler = express.static(__dirname + '/public'); >>>> >>>> app.use(function serveStatic(req, res, next) { >>>> setTimeout(function() { >>>> staticHandler(req, res, next); >>>> }, 500); >>>> }); >>>> >>>> var server = app.listen(3000, function() { >>>> console.log('Listening on port %d', server.address().port); >>>> }); >>>> >>>> >>>> * haproxy config >>>> >>>> global >>>> daemon >>>> quiet >>>> maxconn 1024 >>>> pidfile haproxy.pid >>>> log 127.0.0.1 local0 >>>> log 127.0.0.1 local1 notice >>>> >>>> defaults >>>> log global >>>> >>>> balance roundrobin >>>> mode http >>>> >>>> >>>> frontend external >>>> bind :80 >>>> default_backend test >>>> >>>> backend test >>>> server test localhost:3000 >>>> >>>> >>>> >>>> >>> >>> >>> Hi William, >>> >>> Could you please turn on option httplog and provide us the logs reported by >>> HAProxy? >>> Also, which version of HAProxy are you running? >>> >>> Baptiste >> >> >> Hi William, >> >> And, if possible, could you also please provide PCAP dumps for both scenarios >> >> 1. from both sides of haproxy >> 2. between IOS and your backend server >> >> ? >> >> -- >> Benjamin Lee mailto:[email protected] >> Melbourne, Australia http://www.realthought.net >> Linux / BSD / GNU tel:+61 4 16 BEN LEE >> <Screen Shot 2014-09-29 at 20.08.02.png> > >
