It's not clear to me how to solve this problem. Synopsis: - projects can configure their web server to send files in a compressed form to clients that can handle it; see http://boinc.berkeley.edu/trac/wiki/FileCompression In modern (>= 5.4) BOINC clients, the client tells libCurl to handle such compression. - The client keeps track of the bytes transferred N, and if a transfer fails and is retried, it sends a Range: header telling the web server to skip the first N bytes. NOTE: N is a count of uncompressed bytes; as far as I can tell, libCurl tells us only how many uncompressed bytes it's read. But when send Range:N to the server, it skips N compressed bytes, which causes the error Kevin points out below.
I can think of 3 solutions, 2 of which I don't know how to implement: 1) If a file is compressed, always start the transfer from the beginning. Problem: how does the client know if a file is compressed? This is negotiated between libCurl and the server. 2) Keep track of the number M of compressed bytes transferred, and tell the server to skip M bytes on retries. Problems: a) libCurl doesn't tell us M, as far as I can tell. b) I'm not sure that Apache supports this. 3) Allow a <no_partial_transfer> flag in <file_info> elements, telling the client to always start transfers from the beginning. Does anyone know how to implement 1) or 2), or have other ideas? If not, should I go ahead and implement 3)? -- David -------- Original Message -------- Subject: [BOINC] #924: Partial Transfer Fails Date: Mon, 22 Jun 2009 15:59:20 -0000 From: BOINC <[email protected]> Reply-To: [email protected] #924: Partial Transfer Fails -----------------------------+---------------------------------------------- Reporter: knreed | Owner: davea Type: Defect | Status: new Priority: Major | Milestone: 6.10 Component: Client - Daemon | Version: 6.6.31 Keywords: | -----------------------------+---------------------------------------------- If a project compresses files and relies on libCurl to decompress the file during transfer, then in the event that a transfer is interrupted, BOINC relies on the decompressed file size to set the Request Range header. This will be invalid and therefore fail with: 416 Requested Range Not Satisfiable this will repeat until the timeout limit for the download. It would be great if in the event of the failure of a partial file download, the client would then attempt a full download. -- Ticket URL: <http://boinc.berkeley.edu/trac/ticket/924> BOINC <http://boinc.berkeley.edu> Berkeley Open Infrastructure for Network Computing (BOINC) _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
