Thank you for this so far - this is really useful. Is the config setting - 'network.http.max-persistent-connections-per-server' - is that as per the proxy setting? (or is there some good documentation on what these values stand for)
Here's my understanding/guess at what these actually mean: (some of which from some digging on google - http://kb.mozillazine.org/Network.http.max-persistent-connections-per-proxy) *) network.http.max-connections - total number of HTTP connections, of any type, across any one instance of Firefox (that includes all tabs/windows spawned by that instance) - i.e. a funky web page with 'real-time' polling of http in one tab, would necessarily be in contention with another tab using HDS. (some people have silly numbers of tabs open at once, I know I do!) *) network.http.max-connections-per-server - any one server, identified by the FQDN prefix (e.g. stats.bbc.co.uk is considered different from www.bbc.co.uk), can open HTTP connections up to this maximum (assuming the max-connections hasn't been reached) *) network.http.max-persistent-connections-per-proxy - if a proxy is configured within Firefox (may include a system proxy configuration) then this value controls the number of HTTP connections of type 'keep-alive'. This value is still limited by max-connections-per-server and, in turn, max-connections. [not sure if this is also limited by max-persistent-connections-per-server, though the numbers you gave seem to suggest this isn't - maybe it's possible to have max-persistent-connections-per-proxy > max-persistent-connections-per-server) *) network.http.max-persistent-connections-per-server - this value controls the number of HTTP connections of type 'keep-alive'. (When Firefox is not configured with a proxy). This value is still limited by max-connections-per-server and, in turn, max-connections. Other things that come to mind... *) Fragment URLs may be reused by other copies of the Flash 'HDS' client - under browser based caching rules. Though not clear on how this works - likely the headers on the fragments saying how long they live (not sure, haven't sniffed those yet) *) For the 'http.max...' configuration settings and current 'pipeline' usage - it would be nice to see a couple of things live in Firefox: a) the actual usage of the different HTTP slots for a particular tab/window b) the total for the entire browser - this sounds like something difficult to see, but given the general 'add-on' world of Firefox... is that something that could be exposed? (not for public consumption perhaps) *) Background Tabs - [theory] seem to throttle bandwidth, likely to reduce impact on foreground tabs, resulting in Flash HDS players that employ an adaptive bitrate algorithm to fall back to the lowest bitrate available. Haven't tried this with windows in the background (without focus) - wonder if that is also doing some intelligent throttling. (and when I say 'throttle' - perhaps it's simply that foreground tabs/windows get more priority to the available http connections or that the transfers of the HTTP requests are slower to avoid resource hogging) *) I wonder if pipelining happens on the HDS fragment requests - is there a 'smoking gun' that makes this easy to spot. I'm assuming we'd prefer pipelining, right? *) Seems like these http.max... settings vary between Firefox versions, do they vary on platform? (what's the 'size' of the 'problem', should I be more concerned) *) I am trying to speak to our contacts in Adobe to see what they think about browser cache implications for HDS, and I suspect this will have an impact on their MPEG-DASH thinking too. Do you know if you have had any contact with them around this? *) additional use of SPDY could speed up these sort of fragment requests - though support seems limited. I'd not considered that before. On 19 May 2012 01:28, Steve Workman <[email protected]> wrote: > (Note: lists.mozilla.org seems to be having some issues this week > according to a few colleagues, so these posts might not all appear. Feel > free to email directly if you're concerned). > > On May 18, 2012, at 4:17 AM, Alan Ogilvie wrote: > > > Thank you all for your response so far - clearly this is going to be a > > somewhat complicated answer (I didn't believe that it wouldn't) ;-) > > :) Yup, but we can work it out and de-complexify (new word maybe) it all. > > > > 3/ - yes, my (and somewhat 'our') hypothesis is that because chunk > requests > > are made through the host browser, they will be restricted as any other > > internet content is - for example, requesting a web page, then all the > > assets from the HTML response, is clearly managed by limiting the number > of > > concurrent 'threads' (I may obviously be using the wrong words here) to > > some sort of rule for 'same domain'. If we had more than one player in > the > > browser instance (maybe on multiple tabs/windows) all requesting chunks > > from the same domain, but different 'channels' (ie. different streams), > > then they will likely 'fight' each other for spare slots in the > concurrent > > 'threads'. We're not sure how much of an 'edge case' that is - though I > can > > believe some of our viewers/listeners having a couple of windows open > > during big events (like the Olympics in London coming soon) with HDS > > playback in Flash. > > That's a good example. > > So, I've done so more poking around today, and this is what I can see in > our networking code as far as resources for connections are concerned. > > At a high level, we have 4 maximums for the number of connections (you can > look at these in "about:config" - if you're unfamiliar with that, just type > about:config in the address bar and tell Firefox that you'll be careful): > > network.http.max-connections = 256 > network.http.max-connections-per-server = 15 > network.http.max-persistent-connections-per-proxy = 8 > network.http.max-persistent-connections-per-server = 6 > > I'm guessing that proxy will matter for those folk watching the games from > their workplaces, but there may be other scenarios. And of course, we're > not limited to the Olympic games here. > > These values are configurable, but for the vast majority of users, they > won't touch them. > > Once we get an HTTP request, we create a transaction and determine what to > do next based on the following logic in > nsHttpConnectionMgr::ProcessNewTransaction (currently found at > http://mxr.mozilla.org/mozilla-central/source/netwerk/protocol/http/nsHttpConnectionMgr.cpp#1641). > Note that the transaction is created by the content thread and then > dispatched to the socket thread for the following: > > // 0 - If this should use spdy then dispatch it post haste. > // 1 - If there is connection pressure then see if we can pipeline this > on > // a connection of a matching type instead of using a new conn > // 2 - If there is an idle connection, use it! > // 3 - if class == reval or script and there is an open conn of that > type > // then pipeline onto shortest pipeline of that class if limits > allow > // 4 - If we aren't up against our connection limit, > // then open a new one > // 5 - Try a pipeline if we haven't already - this will be unusual > because > // it implies a low connection pressure situation where > // MakeNewConnection() failed.. that is possible, but unlikely, due > to > // global limits > // 6 - no connection is available - queue it > > So, deciding what to do with the transaction depends on what protocol it's > using (SPDY, HTTP pipelining or non-pipelining), how many connections are > free and the type of connection. If the connection is SPDY, the > transaction is added as a stream if there is room (max of 100 streams per > connection I think). Otherwise the transaction is queued until a slot opens > up in the SPDY stream. So there are multiple layers in the decision-making > to determine what is a relevant resource (num of sockets, num of streams, > etc). > > Some points: > "under pressure" means most of the allowed parallel connections are > already used up. > "idle connections" are persistent connections, i.e. sockets that are kept > open thereby avoiding the DNS and TCP RTT for subsequent requests. > > Once we figure out what type of connection, the transaction is dispatched. > > Some things I'm still exploring: > - We have one socket thread on which the previous decision about how to > add a transaction is processed. I'm not sure if we have more socket threads > for reading and writing or how we multiplex all of that - I'll find that > out. > - There may be a max for the number of requests which can be pipelined. > > > > > I have supplied an example of HDS through Flash which is live on our site > > at the moment. This may help you see how it's working in the real world. > > I've already poked around to see what happens to the chunks. > > What have you observed if anything? > > Steve. > > _______________________________________________ dev-media mailing list [email protected] https://lists.mozilla.org/listinfo/dev-media
