Lars Volker has posted comments on this change. Change subject: IMPALA-4223: Handle truncated file read from HDFS cache ......................................................................
Patch Set 1: (1 comment) > (1 comment) > > Can we test this by having the test load metadata and then truncate > a cached file? We can try to do this in a custom cluster test. It needs to follow the steps outlined here: https://issues.cloudera.org/browse/IMPALA-4223?focusedCommentId=212776&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-212776 They require changes to the system limits to allow for larger cached files, and to the data nodes to increase the file cache as well, so they might be rather disruptive. Once these are changed we can write a custom cluster test to download, truncate, and upload files and run queries over them, checking that the correct log messages appear. Should we break this out into several Jiras / changes? The limits should be change in impala-setup, too. The datanode settings change will be required to integrate this into fuzz testing, too. I will also have a look at the scanner fuzz test and see if it is easy to inject these error there. http://gerrit.cloudera.org:8080/#/c/4828/1/be/src/runtime/disk-io-mgr-scan-range.cc File be/src/runtime/disk-io-mgr-scan-range.cc: PS1, Line 448: disk > how about "uncached" instead? I followed the error from L433, assuming that since we already report a similar case it made sense to be consistent. Should I change both? -- To view, visit http://gerrit.cloudera.org:8080/4828 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Id1e1fdb0211819c5938956abb13b512350a46f1a Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
