[ https://issues.apache.org/jira/browse/HDFS-14083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687729#comment-16687729 ]
Steve Loughran commented on HDFS-14083: --------------------------------------- I've moved it to HDFS, deleted the fix version as that only goes in when the fix is there; "target version" is the one to put the one you want in. w.r.t testing, the purpose of tests is not to just to show that you've fixed an issue, *but to make sure it nevers come back*. Yes, coming up with a good test is hard Yes, it often involves more code than the bit of the patch which goes into production And the standards for test code are high too But: its how we improve the codebase and keep the quality high. My personal view is "the sole excuse for not writing a test is that its impossible to either replicate the situation or to verify that the solution actually worked", though that's where mockito comes into its own. > libhdfs logs errors when opened FS doesn't support ByteBufferReadable > --------------------------------------------------------------------- > > Key: HDFS-14083 > URL: https://issues.apache.org/jira/browse/HDFS-14083 > Project: Hadoop HDFS > Issue Type: Improvement > Components: native > Affects Versions: 3.0.3 > Reporter: Pranay Singh > Assignee: Pranay Singh > Priority: Minor > Attachments: HADOOP-15928.001.patch, HADOOP-15928.002.patch > > > Problem: > ------------ > There is excessive error logging when a file is opened by libhdfs > (DFSClient/HDFS) in S3 environment, this issue is caused because buffered > read is not supported in S3 environment, HADOOP-14603 "S3A input stream to > support ByteBufferReadable" > The following message is printed repeatedly in the error log/ to STDERR: > {code} > -------------------------------------------------------------------------------------------------- > UnsupportedOperationException: Byte-buffer read unsupported by input > streamjava.lang.UnsupportedOperationException: Byte-buffer read unsupported > by input stream > at > org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:150) > {code} > h3. Root cause > After investigating the issue, it appears that the above exception is printed > because > when a file is opened via {{hdfsOpenFileImpl()}} calls {{readDirect()}} which > is hitting this > exception. > h3. Fix: > Since the hdfs client is not initiating the byte buffered read but is > happening in a implicit manner, we should not be generating the error log > during open of a file. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org