[
https://issues.apache.org/jira/browse/MAPREDUCE-6978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198051#comment-16198051
]
rangjiaheng commented on MAPREDUCE-6978:
----------------------------------------
Thank you, Jason !
We solved this problem simply by adding an judgment after read ordinal, but if
someone add an Enumeration value in the middle position of TaskCounter.class,
the AM will read confused Enumeration values from RPC.
I support counters should serialized through protocol buffer.
{code:java}
public void readFields(DataInput in) throws IOException {
clear();
int len = WritableUtils.readVInt(in);
T[] enums = enumClass.getEnumConstants();
for (int i = 0; i < len; ++i) {
int ord = WritableUtils.readVInt(in);
if(ord >= enums.length) {
WritableUtils.readVLong(in);
continue;
}
Counter counter = newCounter(enums[ord]);
counter.setValue(WritableUtils.readVLong(in));
counters[ord] = counter;
}
}
{code}
> MR task counters deserialized through RPC throws OutOfBoundsException if
> Counter enum class version not match
> -------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6978
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6978
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mr-am, task
> Affects Versions: 3.0.0-alpha4
> Environment: NM1 TaskCounter.class old version;
> NM2 TaskCounter.class new version (new Enumeration values appended);
> Reporter: rangjiaheng
>
> Environment:
> NM1 TaskCounter.class old version;
> NM2 TaskCounter.class new version (new Enumeration values appended);
> Result:
> When an MR app's AM running on NM1, and it's containers on NM2; the
> containers on NM2 will all failed, AM cause OutOfBoundsException;
> Reason:
> When app running, containers will report their counters to AM through RPC,
> while the Container with new version TaskCounter.class will write more
> Counter values to RPC; however, the AM with old version TaskCounter.class
> which can not read them correctly from RPC.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
