Hi, TL;DR:
I've been experimenting with HTTP/2 Push and Prefetch to preposition content on the client before it requests it, and specifically I want to evaluate the pitfalls of prepositioning objects aggressively, with regards to how it could impact the delivery of higher priority objects. While experimenting with post-onLoad prepositioning mechanisms to optimize next page navigations (mostly with prefetch/prerender), I noticed that pending prepositioned objects can considerably delay the next page; if the next page navigation is triggered while prepositioned objects are in flight, the next page HTML will be backed behind a considerable part of them on the server. >From what I can tell, object/stream priorities cannot help since the HoL blocking happens at the SSL socket where the server can't preempt data that it has already written. So I wonder if there could be some logic on the server that controls how aggressively it writes low-priority frames to the SSL socket, making sure they are being drained before pushing more and bloating the buffer (and causing critical objects that are requested soon after to be delayed). Details: To evaluate the impact, I tried a simple, extreme scenario where I trigger the Prefetch (low-priority) of a large image after the onLoad of page A fires, and soon after it fires and the image is being downloaded I manually navigate to page B. In this case it takes as long as it takes for the image to download before the next page starts showing. That was many seconds in my case since it was a purposefully large (5MB) image and bad connection (4G, ~200ms from server). I also tried prefetching 8 copies of the same image, over a fast connection (cable, 20ms to the server). I navigate to page B while the first few prefetched images are being downloaded. In this case page B's HTML arrived 900ms after the request was sent. Looking at Chrome's event log, the HTML (stream weight 256) was indeed prioritized over the prefetched images (weight 110): the HTML arrives in the middle of the 4th prefetched image - but there was still more than 1xBDP worth of data that arrived during the 900ms between the HTML request-response. I believe that these are H2 frames that have been written for sending (but not yet sent on the wire), where the server can no longer preempt them so anything new (regardless of priority) can't be sent on the wire before the buffer is drained. My hypothesis was that priorities are honored at the application level - i.e. the HTML frames are interleaved into the socket as soon as the request arrives (I think I've now verified that from both server and chrome logs) - but the socket doesn't keep up at draining the previously written low-prio frames. I've also observed that the HTML's delay is in multiples of RTTs. So I guess I'm just describing an instance of bufferbloat, which is likely to happen when many/large low-prio requests arrive at the server before a critical one (as is the case with prepositioning) So I think some Head-of-LIne blocking (which H2 is supposed to do away with) is happening even though priorities are honored in the application layer. As a result the client receives the critical object (much) later than it could have. I considered tweaking the browser to open a second connection for prefetch/pushed/otherwise low-prio content, but this goes against the goals/purpose of HTTP/2. I believe that the right place to tweak this would be the server. I think the application does things right as far as it knows, i.e. frames leave the application layer in OK priority, but not the wire. What do you think would be the right way to fix this (assuming it is indeed something that needs fixing)? Do you think there could be some additional logic that controls how aggressively the server writes to the SSL socket, that checks how much of low-prio data has been drained from the socket before writing more, to avoid bloating the buffer on the server side with low-priority frames which more critical frames will have to wait behind when they become available? I've started looking at httpd's source code to get a sense of how big of a change this would be, but since I haven't looked at the code before I wanted to get your thoughts on whether this makes sense first. Thanks and sorry for the length