[
https://issues.apache.org/jira/browse/HDFS-4403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13555542#comment-13555542
]
Colin Patrick McCabe commented on HDFS-4403:
--------------------------------------------
{code}
* @param src The file path
* @return The checksum
*/
- public static MD5MD5CRC32FileChecksum getFileChecksum(String src,
+ static MD5MD5CRC32FileChecksum getFileChecksum(String src,
+ String clientName,
{code}
Need some JavaDoc about the new parameter, {{clientName}}.
{code}
- final DataChecksum.Type ct = PBHelper.convert(checksumData
- .getCrcType());
+ DataChecksum.Type ct;
...
{code}
For what it's worth, {{ct}} can still be {{final}} here (it only gets set once).
I realize this isn't new with your change, but I find it kind of odd that the
size of our *output* {{BufferedOutputStream}} is derived from the size of
{{io.file.buffer.size}}. Surely, these requests are small, and there's no
reason to allocate a giant send buffer just because we wanted more file
buffering? Plus these are not "file buffers"-- we're not sending file data.
> DFSClient can infer checksum type when not provided by reading first byte
> -------------------------------------------------------------------------
>
> Key: HDFS-4403
> URL: https://issues.apache.org/jira/browse/HDFS-4403
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 2.0.2-alpha
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: hdfs-4403.txt
>
>
> HDFS-3177 added the checksum type to OpBlockChecksumResponseProto, but the
> new protobuf field is optional, with a default of CRC32. This means that this
> API, when used against an older cluster (like earlier 0.23 releases) will
> falsely return CRC32 even if that cluster has written files with CRC32C. This
> can cause issues for distcp, for example.
> Instead of defaulting the protobuf field to CRC32, we can leave it with no
> default, and if the OpBlockChecksumResponseProto has no checksum type set,
> the client can send OP_READ_BLOCK to read the first byte of the block, then
> grab the checksum type out of that response (which has always been present)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
