[
https://issues.apache.org/jira/browse/HADOOP-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew Paduano updated HADOOP-12563:
-------------------------------------
Attachment: HADOOP-12563.14.patch
using writeDelimitedTo/readDelimitedFrom in the proto IO:
{code}
--- a/HADOOP-12563.13.patch
+++ b/HADOOP-12563.14.patch
@@ -322,7 +322,7 @@ index e6b8722..662eb3e 100644
+ setSecret(ByteString.copyFrom(e.getValue()));
+ storage.addSecrets(kv.build());
+ }
-+ storage.build().writeTo((DataOutputStream)out);
++ storage.build().writeDelimitedTo((DataOutputStream)out);
+ }
+
/**
@@ -331,7 +331,7 @@ index e6b8722..662eb3e 100644
+ * @param in - stream ready to read a serialized proto buffer message
+ */
+ public void readProtos(DataInput in) throws IOException {
-+ CredentialsProto storage = CredentialsProto.parseFrom((DataInputStream)in)
++ CredentialsProto storage = CredentialsProto.parseDelimitedFrom((DataInputS
+ for (CredentialsKVProto kv : storage.getTokensList()) {
+ addToken(new Text(kv.getAliasBytes().toByteArray()),
+ (Token<? extends TokenIdentifier>) new Token(kv.getToken()));
{code}
> Updated utility to create/modify token files
> --------------------------------------------
>
> Key: HADOOP-12563
> URL: https://issues.apache.org/jira/browse/HADOOP-12563
> Project: Hadoop Common
> Issue Type: New Feature
> Affects Versions: 3.0.0
> Reporter: Allen Wittenauer
> Assignee: Matthew Paduano
> Fix For: 3.0.0
>
> Attachments: HADOOP-12563.01.patch, HADOOP-12563.02.patch,
> HADOOP-12563.03.patch, HADOOP-12563.04.patch, HADOOP-12563.05.patch,
> HADOOP-12563.06.patch, HADOOP-12563.07.patch, HADOOP-12563.07.patch,
> HADOOP-12563.08.patch, HADOOP-12563.09.patch, HADOOP-12563.10.patch,
> HADOOP-12563.11.patch, HADOOP-12563.12.patch, HADOOP-12563.13.patch,
> HADOOP-12563.14.patch, dtutil-test-out,
> example_dtutil_commands_and_output.txt, generalized_token_case.pdf
>
>
> hdfs fetchdt is missing some critical features and is geared almost
> exclusively towards HDFS operations. Additionally, the token files that are
> created use Java serializations which are hard/impossible to deal with in
> other languages. It should be replaced with a better utility in common that
> can read/write protobuf-based token files, has enough flexibility to be used
> with other services, and offers key functionality such as append and rename.
> The old version file format should still be supported for backward
> compatibility, but will be effectively deprecated.
> A follow-on JIRA will deprecrate fetchdt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
