https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30614
David Cook <[email protected]> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |[email protected] --- Comment #2 from David Cook <[email protected]> --- When it comes to checking URLs, typically you want to do a HTTP HEAD instead of a HEAD GET to increase performance for the checker and to decrease load on the URL being checked. For instance, let's say you're checking if URL "https://path/to/1TB/file.file"; exists. If you do a HTTP HEAD, that can be a <1 second check. If you do a HTTP GET, you're going to have to wait for that file to download, so your checker will go much slower. It's also going to cost the server host money in terms of data transfer. If you're using cloud like Azure or AWS, your costs can increase by hundreds or thousands of dollars very easily. So HTTP HEAD is much better than HTTP GET for this use case. However... not all sites honour HTTP HEAD. Misguided people trying to lockdown servers for security reasons will sometimes limit HTTP verbs to just GET/POST, but this actually has a negative effect, because it means you then have to do a full HTTP GET to do something like a URL check. I work on a project with millions of URLs and many terabytes of data, and HTTP HEAD is one of my best friends. Anyway... I'd say an improvement to the script would be to add a CLI parameter to choose whether to use a "HEAD" or a "GET" request, because realistically sometimes you do have to use a GET to get an accurate response - even if HEAD is more optimal in an ideal world. Another optimisation could be to fall back to a GET in the event that a HEAD fails. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes. _______________________________________________ Koha-bugs mailing list [email protected] https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
