[
https://issues.apache.org/jira/browse/HADOOP-6170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756167#action_12756167
]
Philip Zeyliger commented on HADOOP-6170:
-----------------------------------------
I took a look. Overall, looks good. I'm sufficiently behind on my Avro-fu
that I
had a hard time following some of the magic.
I'm writing down below what my current understanding is. Let me know
where I'm totally wrong. Do add some JavaDoc per class.
TunnelProtocol is the meta-interface that's used by Hadoop IPC.
BufferListWritable is used by Hadoop IPC to ship
the Avro byte-stream around. Its serialization format is:
{code}
data := size (buffer)[size]
buffer := len (bytes)[len]
size, len are writable integers
bytes are raw bytes
x[count] indicates count repetitions of x.
{code}
ClientTranceiver delegates shipping of bytes via Hadoop IPC.
ServerTranceiver doesn't do much of anything except store
the response data. New instances of it are created at every
call in TunnelResponder.
TunnelResponder implements the Hadoop IPC (TunnelProtocol);
it's the thing running on the server, and converts calls
into Avro RPC calls. This class would be clearer if, instead
of extending ReflectResponder, you kept a private ReflectResponder,
and called that explicitly in call. Then you would rename
TunnelResponder to TunnelProtocolImpl.
Can a single RPC server satisfy multiple protocols? Does that
work with AvroRPC? I don't think it can right now, but I think
that's necessary, since several daemons implement a handful of
protocols.
Some specific notes:
bq. versioning features to for inter-Java RPCs.
Typo.
AvroTestProtocol is generated code, yes? Or does ReflectData.getProtocol()
figure it out from reflection and paranamer data? If the former,
should the source schema be checked in? Should the generation
be done in build.xml?
bq. assertEquals(intResult, 3);
Total nit: JUnit prefers the expected argument on the left.
bq. public BufferListWritable() {}
Eclipse tells me this is unused. Might be worth
a quick comment indicating that it's required because
Writables get instantiated via reflection.
bq. getRemoteName() { return "remote"; }
Should these be customized to the Protocol being used?
Cheers,
-- Philip
> add Avro-based RPC serialization
> --------------------------------
>
> Key: HADOOP-6170
> URL: https://issues.apache.org/jira/browse/HADOOP-6170
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Fix For: 0.21.0
>
> Attachments: HADOOP-6170.patch, HADOOP-6170.patch, HADOOP-6170.patch
>
>
> Permit RPC protocols to use Avro to serialize requests and responses, so that
> protocols may better evolve without breaking compatibility.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
