Dan Hecht has posted comments on this change. Change subject: IMPALA-4223: Handle truncated file read from HDFS cache ......................................................................
Patch Set 1: (1 comment) > (1 comment) > > > (1 comment) > > > > Can we test this by having the test load metadata and then > truncate > > a cached file? > > We can try to do this in a custom cluster test. It needs to follow > the steps outlined here: > https://issues.cloudera.org/browse/IMPALA-4223?focusedCommentId=212776&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-212776 > > They require changes to the system limits to allow for larger > cached files, and to the data nodes to increase the file cache as > well, so they might be rather disruptive. Once these are changed we > can write a custom cluster test to download, truncate, and upload > files and run queries over them, checking that the correct log > messages appear. > > Should we break this out into several Jiras / changes? The limits > should be change in impala-setup, too. The datanode settings change > will be required to integrate this into fuzz testing, too. > > I will also have a look at the scanner fuzz test and see if it is > easy to inject these error there. Yes, if it's a good amount of work to set up that kind of test, let's save it for another step. It's probably better to spend time with getting the fuzzer working rather than this one particular test case. http://gerrit.cloudera.org:8080/#/c/4828/1/be/src/runtime/disk-io-mgr-scan-range.cc File be/src/runtime/disk-io-mgr-scan-range.cc: PS1, Line 448: disk > I followed the error from L433, assuming that since we already report a sim Okay. I'm fine either leaving this one as "disk" or changing both. -- To view, visit http://gerrit.cloudera.org:8080/4828 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Id1e1fdb0211819c5938956abb13b512350a46f1a Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Lars Volker <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
