On 25.07.12 14:49, [email protected] wrote: > Have the validator mark the files as read only as the first step in what it > does.
Hm. If that would work, it would also prevent the file_deleter from deleting the file later. Then either the assimilator needs to make the file writeable again after assimilation, or the file_deleter needs to use more 'force' to delete the file. And what would the (current) file_upload_handler do if it finds it can't write the file? I prefer fixing the root cause of problems over treating symptoms. IMHO the root cause here is that the file_upload_handler accepts any file any time, overwriting whatever may be already there (and e.g. already validated). FWIW we found a few more assimilated files (for a different application) that could never have made it through the validator that was running at that time. There is a slight chance of a hardware defect on our server RAID that keeps the assimilated results, but there is no other indication for that - no other files are broken, no SMART errors etc. Unfortunately the file_upload_handler logs of that time were already deleted (we keep these only for 200d). Best, Bernd > Hi Kevin! > > On 24.07.12 23:38, Kevin Reed wrote: > >> I think that the result file record includes a md5sum that you could > check >> during validation and assimilation to confirm that it is the same file. >> This would prevent improper assimilation. > Ok, I'll check this. > > I'm a bit worried, though, about the additional I/O load. Our current > assimilator doesn't even read the result files, it just manipulates > metadata > (hardlinks). And I would prefer a solution that _prevents_ the file to be > modified after validation over something that just _detects_ a possible > modification. What should the assimilator do if it finds the file has been > modified since validation? > >> How often have you seen this? > There is one case where we noticed it, because it happened in the canonical > results of a certain application. I'm still digging through the logs to > find cases where this might have gone unnoticed. > > This particular case involved a client behaving really mad, initiating an > upload of a file with the same name (but four different reported file > sizes) > 221 times in total, up to 7 times in a single second, and apparently even > after reporting the result. (Uploading a result file after the result was > reported (once) is probably not uncommon - imagine a BOINC directory > restored from a backup.) > > I don't expect to find many such cases, but the number of these is rather > irrelevant. To me it shows a serious (design) problem that this could > happen > at all. Every possible check for scientific validity should be done in the > validator, under _no_ circumstances should the server allow _any_ client to > > modify a result _after_ validation, no matter how stupid or rogue the > client behaves. > > Best, > Bernd > > _______________________________________________ > boinc_dev mailing list > [email protected] > http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev > To unsubscribe, visit the above URL and > (near bottom of page) enter your email address. > > > _______________________________________________ boinc_dev mailing list [email protected] http://lists.ssl.berkeley.edu/mailman/listinfo/boinc_dev To unsubscribe, visit the above URL and (near bottom of page) enter your email address.
