Mustafa İman created HIVE-24415: ----------------------------------- Summary: HiveSplitGenerator blocks Tez dispatcher Key: HIVE-24415 URL: https://issues.apache.org/jira/browse/HIVE-24415 Project: Hive Issue Type: Improvement Reporter: Mustafa İman Assignee: Mustafa İman
HiveSplitGenerator does a lot of heavyweight operations in its constructor. These operations block AsyncDispatcher in Tez [https://github.com/apache/tez/blob/989d286d09cac7c4e4c5a0e06dd75ea5a6f15478/tez-dag/src/main/java/org/apache/tez/dag/app/dag/RootInputInitializerManager.java#L141] . We should move bulk of initialization out of constructor. The only reason of setting up everything in constructor is DynamicPartitionPruner. We can buffer incoming events in HiveSplitGenerator until dynamic partition pruner is initialized. -- This message was sent by Atlassian Jira (v8.3.4#803005)