Jingsong Lee created FLINK-14722: ------------------------------------ Summary: Optimize mapred.HadoopInputSplit to not serialize conf when split is not configurable Key: FLINK-14722 URL: https://issues.apache.org/jira/browse/FLINK-14722 Project: Flink Issue Type: Improvement Components: Connectors / Hadoop Compatibility Reporter: Jingsong Lee Fix For: 1.10.0
JobConf may very big, contains hundreds of configurations, if it is serialized by every split, that will significantly reduce performance. Consider thousands of splits, the akka thread of JobMaster will all on the serialization of conf. That may will lead to various akka timeouts too. -- This message was sent by Atlassian Jira (v8.3.4#803005)