================================= SUMMARY ======================================
My public fork of Wget2 project is available here [0]. I will continuously keep
pushing my work so anyone interested can track me there. Feel free to
participate in the discussions going on merge request with my mentors [1]. Your
feedback are highly appreciated.
=============================== INTRODUCTION ===================================
The purpose of this project is to use Libmicrohttpd as test suite for Wget2. I
plan to do this by do some changes on function wget_test_start_server() also
wget_test_stop_server() on src/libtest.c of Wget2. With this approach, I don't
need to change existing test suite which call the internal server code through
functions mentioned above.
I've count there are 33 test file which use wget_test_start_server(). I must
ensure all the test passed when I've done the transition.
For installation prerequisite, I make that Libmicrohttpd support will needed
when we need to run `make check` or run test suite. Then I need to modify
configure.ac. I will give proper warning about this requirement. There is a
section in README.md where I must explain to user to provide Libmicrohttpd to
make all test running correctly.
With Libmicrohttpd I can add new test using feature that not yet implemented in
old server code, but ready on Libmicrohttpd, such as HTTP digest
authentication.
Mentors:
Darshit Shah <dar...@gmail.com>
Ander Juaristi <ajuari...@gmx.es>
================================== UPDATES =====================================
Things which were done until this week:
* Started working on wget_test_start_server() to use Libmicrohttpd as HTTP
server. Workflow to resolve this:
- Remove initial process for HTTP server socket
- Create _http_server_start() function, wrapper for Libmicrohttpd. There is
also function _answer_to_connection() which use to create proper HTTP
response
- Use select method (MHD_USE_SELECT_INTERNALLY) for threading model in
Libmicrohttpd to get better compatibility
- http_server_port seized automatically using Libmicrohttpd function by
passing MHD_DAEMON_INFO_BIND_PORT or MHD_DAEMON_INFO_LISTEN_FD parameter to
MHD_get_daemon_info()
- Using iteration to parse urls data in _answer_to_connection(). This
guarantee we can pass any variadic data to Libmicrohttpd and prevent
segmentation fault
- Fix _answer_to_connection() function to create proper HTTP response:
- Handle arguments (query string) on URL
- Handle redirection
- Handle directory creation
- Handle URL with IRI object
- Handle URL with IDN hostname
- Handle URL with query string which contains space
- Handle If-Modified-Since header
- Handle Byte Range request
- Handle HTTP basic authentication
- Handle HTTPS
- Handle chunked transfer encoding Notes: I use
MHD_create_response_from_callback() from Libmicrohttpd to generate
chunked transfer method. For simulation with bad chunked data, I use
MHD_create_response_from_buffer() with proper header and use connection
close. I think the result of this method is most closely approached with
the result from old HTTP server
* Fix compiler warning -Wunused-param
* Use persistent connection (Connection: keep-alive) instead multiple
connection (Connection: close). This prevent hiding several problems of the
connection between HTTP server and client
* Fix CI testing on Travis OSX: must install Libmicrohttpd from source to get
support for HTTPS. Maybe the Libmicrohttpd package provided by brew not
compiled with HTTPS support
* Fix CI testing on Gitlab MinGW64. That problem caused by variable clash
between Wget2 and MinGW64 header have been reported and fixed
* Add CI testing to simulate Wget2 build without Libmicrohttpd installed. This
is just a copy of main build process but without 'make check' part, so the
Wget2 binary can be built with skipped testing [2][3]
================================= NEXT STEPS ===================================
Things which would be done in the coming week:
* Add HTTP digest authentication test
* Add other tests which seems to possible to add in this time frame
(compression test, IPv6 test, etc). I need more suggestion about this
[0]: https://gitlab.com/gnuwget/wget2/tree/tmp-use-mhd
[1]: https://gitlab.com/gnuwget/wget2/merge_requests/265
[2]: https://gitlab.com/dstw/wget2/tree/mhd-not-install
[3]: https://github.com/dstw/wget2/tree/mhd-not-install
Regards,
Didik Setiawan
