[ 
https://issues.apache.org/jira/browse/HADOOP-10704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suraj Nayak M updated HADOOP-10704:
-----------------------------------

    Description: 
A file is created using LocalFileSystem. When the file is corrupted/changed and 
tried to read, it throws ChecksumException as the CRC does not match. But this 
process does not move the affected file to "bad_files" directory.

Code : 
public void testReportChecksumFailureOnCorruption() throws IOException {
                String sampleData = "My Test Data. This will be corrupted soon";
                FileSystem fs = LocalFileSystem.get(new Configuration());
                Path rootPath = new Path(TEST_ROOT_DIR);
                FSDataOutputStream fsOutStream = fs.create(new Path(rootPath
                                + "/testFile.txt"));
                ByteArrayInputStream inStream = new ByteArrayInputStream(
                                sampleData.getBytes());
                IOUtils.copyBytes(inStream, fsOutStream, 4096, false);
                fsOutStream.close();
                inStream.close();
                // Corrupt the file
                File corruptFile = new File(TEST_ROOT_DIR + "/testFile.txt");
                FileOutputStream fOutCorruptingStream = new FileOutputStream(
                                corruptFile, true);
                fOutCorruptingStream.write(". This is bad data!".getBytes());
                fOutCorruptingStream.close();
                // Read the corrupted file using LocalFS
                FileSystem readLFS = LocalFileSystem.get(new Configuration());
                FSDataInputStream fsInStream = readLFS.open(new 
Path(TEST_ROOT_DIR
                                + "/testFile.txt"));
                try {
                        byte[] b = new byte[10];
                        fsInStream.read(b, 0, 10);
                } catch (Exception e) {
                        if (corruptFile.exists()) {
                                // Should not exit before moving the corrupt 
file to bad_files
                                assertFalse(true);
                        } else {
                                // Should exit after moving the corrupt file to 
bad_files
                                assertTrue(true);
                        }
                }
                fsInStream.close();
        }

  was:A file is created using LocalFileSystem. When the file is 
corrupted/changed and tried to read, it throws ChecksumException as the CRC 
does not match. But this process does not move the affected file to "bad_files" 
directory.


> Corrupt Files are not moved to bad_files in LocalFileSystem
> -----------------------------------------------------------
>
>                 Key: HADOOP-10704
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10704
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>            Reporter: Suraj Nayak M
>            Priority: Minor
>
> A file is created using LocalFileSystem. When the file is corrupted/changed 
> and tried to read, it throws ChecksumException as the CRC does not match. But 
> this process does not move the affected file to "bad_files" directory.
> Code : 
> public void testReportChecksumFailureOnCorruption() throws IOException {
>               String sampleData = "My Test Data. This will be corrupted soon";
>               FileSystem fs = LocalFileSystem.get(new Configuration());
>               Path rootPath = new Path(TEST_ROOT_DIR);
>               FSDataOutputStream fsOutStream = fs.create(new Path(rootPath
>                               + "/testFile.txt"));
>               ByteArrayInputStream inStream = new ByteArrayInputStream(
>                               sampleData.getBytes());
>               IOUtils.copyBytes(inStream, fsOutStream, 4096, false);
>               fsOutStream.close();
>               inStream.close();
>               // Corrupt the file
>               File corruptFile = new File(TEST_ROOT_DIR + "/testFile.txt");
>               FileOutputStream fOutCorruptingStream = new FileOutputStream(
>                               corruptFile, true);
>               fOutCorruptingStream.write(". This is bad data!".getBytes());
>               fOutCorruptingStream.close();
>               // Read the corrupted file using LocalFS
>               FileSystem readLFS = LocalFileSystem.get(new Configuration());
>               FSDataInputStream fsInStream = readLFS.open(new 
> Path(TEST_ROOT_DIR
>                               + "/testFile.txt"));
>               try {
>                       byte[] b = new byte[10];
>                       fsInStream.read(b, 0, 10);
>               } catch (Exception e) {
>                       if (corruptFile.exists()) {
>                               // Should not exit before moving the corrupt 
> file to bad_files
>                               assertFalse(true);
>                       } else {
>                               // Should exit after moving the corrupt file to 
> bad_files
>                               assertTrue(true);
>                       }
>               }
>               fsInStream.close();
>       }



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to