Hi Hubert!Congrats on your selection. I look forward to a great summer of code in Wget this time around.
On 04/29, Hubert Tarasiuk wrote:
Hello developers, My proposal for *Speed up Wget's Download Mechanism* has been accepted by the mentors! There are two tasks to be done there: - conditional GET requests (if-modified-since) (RFC7232) - TCP Fast Open (RFC7413) A summarized version of my proposal is available: http://pliki.h.trsk.org/gsoc/wget_public.pdf IMHO it is quite obvious how the first feature should be implemented in Wget. However, there is some more moving around needed to use TFO. I have proposed two possible ways in the above PDF. Perhaps you can express your opinion about the approaches, or you have another idea for accomplishing it?
There's two separate points I want to make here:1. With respect to the changes in the Wget source, I think it is saner to merge the connect methods. Just ensure that we can handle proxies and FTP connections without any code duplication. I don't think there should be anything special when making a HTTPS connection? 2. Regarding the socket options, we should spend some more time evaluating our options. My understanding of TCP_CORK is that it may be a useful option for Servers, but it doesn't really affect TCP clients in any useful way. This is because TCP_CORK modifies the minimum TCP packet size by buffering for as much data before sending it out. With the small request sizes that a HTTP client would generally send, I think it is better to follow Nagle's algorithm, since TCP_CORK will not afford us any noticeable advantage. On the other hand, it's non-portability will be a nightmare for us when trying to support OSX, BSD and Windows.
I haven't gone through this code thoroughly yet, but they tried to reproduce the results of the original TFO whitepaper using a Python HTTP Server, like the one we use for our test suite. Maybe we can borrow some code from them?Another issue I am thinking about is how to test the TFO feature. I am not very familiar with network API in Python, but my first idea would be to count the TCP segments sent and received and/or to check that the first packet (with SYN flag) contains data (the request). What do you think?
I will be thankful for any other suggestions, as well. Have a good day, Hubert
-- Thanking You, Darshit Shah
pgpOuc3ZBItct.pgp
Description: PGP signature
