[
https://issues.apache.org/jira/browse/HDFS-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15989654#comment-15989654
]
Chris Douglas commented on HDFS-6984:
-------------------------------------
v006 of the patch does the following:
* Moves all the protobuf serialization to library code
* Deprecates Writable APIs for both {{FileStatus}} and {{FsPermission}}
* Where possible, removes {{FsPermissionExtension}}.
** Too many unit tests rely on this for HDFS, and trying to change those caused
even more bloat to the patch. v006 moves its instantiation to a private
{{convert}} method. If this is OK, I'll file a followup JIRA to clear up the
unit tests.
** I don't know how many downstream applications rely on the {{hasAcl}},
{{isEncrypted}}, or {{isErasureCoded}} methods on {{FsPermission}}, but these
are deprecated (rather than removed) in the patch.
** Introduced an intermediate, private {{FlaggedFileStatus}} class to preserve
the attributes formerly mixed in with the permission bits. This could have been
in {{LocatedFileStatus}}, but downstream clients may check {{instanceof
LocatedFileStatus}} and assume the null locations are correct.
** Still need to deprecate {{FsPermission#toExtendedShort}}, will post a
followup patch with any other checkstyle/findbugs fixes to v006
* Make {{HdfsFileStatus}} extend {{FileStatus}}
** After HADOOP-13895, the {{Serializable}} API bled further into the API
** {{getSymlink}} annoyingly throws an {{IOException}} if {{!isSymlink()}}.
Overriding it in {{HdfsFileStatus}} required changing the return type, and to
comply with the contract tests there are some superfluous try/catch statements
in e.g., the JSON utils
** The JSON code tried to preserve the acl/crypt/ec bits of the
{{FsPermission}} on {{AclStatus}}. This seems incorrect, but I can try to find
a solution if it is meaningful.
> In Hadoop 3, make FileStatus serialize itself via protobuf
> ----------------------------------------------------------
>
> Key: HDFS-6984
> URL: https://issues.apache.org/jira/browse/HDFS-6984
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 3.0.0-alpha1
> Reporter: Colin P. McCabe
> Assignee: Colin P. McCabe
> Labels: BB2015-05-TBR
> Attachments: HDFS-6984.001.patch, HDFS-6984.002.patch,
> HDFS-6984.003.patch, HDFS-6984.004.patch, HDFS-6984.005.patch,
> HDFS-6984.006.patch, HDFS-6984.nowritable.patch
>
>
> FileStatus was a Writable in Hadoop 2 and earlier. Originally, we used this
> to serialize it and send it over the wire. But in Hadoop 2 and later, we
> have the protobuf {{HdfsFileStatusProto}} which serves to serialize this
> information. The protobuf form is preferable, since it allows us to add new
> fields in a backwards-compatible way. Another issue is that already a lot of
> subclasses of FileStatus don't override the Writable methods of the
> superclass, breaking the interface contract that read(status.write) should be
> equal to the original status.
> In Hadoop 3, we should just make FileStatus serialize itself via protobuf so
> that we don't have to deal with these issues. It's probably too late to do
> this in Hadoop 2, since user code may be relying on the existing FileStatus
> serialization there.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
