Yicong Cai created YARN-8691:
--------------------------------
Summary: AMRMClient unregisterApplicationMaster Api's appMessage
should have a maximum size
Key: YARN-8691
URL: https://issues.apache.org/jira/browse/YARN-8691
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.7.3
Reporter: Yicong Cai
Fix For: 2.7.7
SparkSQL AM Codegen ERROR，then call unregister AM API and send the error
message to RM, RM receive the AM state and update to RMStateStore. The Codegen
error message maybe is huge, (Our case is about 200MB). If the RMStateStore is
ZKRMStateStore, it causes the same exception as
[YARN-6125|https://issues.apache.org/jira/browse/YARN-6125], but YARN-6125
doesn't cover the unregisterApplicationMaster's message cut.
SparkSQL Codegen error message show below:
18/08/18 08:34:54 ERROR codegen.CodeGenerator: failed to compile:
org.codehaus.janino.JaninoRuntimeException: Constant pool has grown past JVM
limit of 0xFFFF
/* 001 */ public java.lang.Object generate(Object[] references) {
/* 002 */ return new SpecificSafeProjection(references);
/* 003 */ }
/* 004 */
/* 005 */ class SpecificSafeProjection extends
org.apache.spark.sql.catalyst.expressions.codegen.BaseProjection {
..........
about 2 million lines.
..........
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
