[
https://issues.apache.org/jira/browse/HIVE-24206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhou updated HIVE-24206:
------------------------
Description:
the sql is very long，about 3M size，use hive parse to parse the sql, it costs
more than one hour to finish it. The reason is hive parse use IdentityHashMap,
is not multi-thread safe, than it costs more than one hour to execute the code
"while(true)"
my code is :
ParseDriver pd = new ParseDriver();
ASTNode tree = pd.parse(query, context);
while ((tree.getToken() == null) && (tree.getChildCount() > 0))
{ tree = (ASTNode) tree.getChild(0); }
Map<Rule, NodeProcessor> rules = new LinkedHashMap<>();
Dispatcher disp = new DefaultRuleDispatcher(this, rules, null);
GraphWalker ogw = new DefaultGraphWalker(disp);
List<Node> topNodes = new ArrayList<>();
topNodes.add(tree);
ogw.startWalking(topNodes, null);
was:the sql is very long，about 3M size，use hive parse to parse the sql, it
costs more than one hour to finish it. The reason is hive parse use
IdentityHashMap, is not multi-thread safe, than it costs more than one hour to
execute the code "while(true)"
> DefaultGraphWalker IdentityHashMap causes thread idling
> -------------------------------------------------------
>
> Key: HIVE-24206
> URL: https://issues.apache.org/jira/browse/HIVE-24206
> Project: Hive
> Issue Type: Bug
> Components: Parser
> Affects Versions: 2.3.4
> Environment: my code is :
> ParseDriver pd = new ParseDriver();
> ASTNode tree = pd.parse(query, context);
> while ((tree.getToken() == null) && (tree.getChildCount() > 0)) {
> tree = (ASTNode) tree.getChild(0);
> }
> Map<Rule, NodeProcessor> rules = new LinkedHashMap<>();
> Dispatcher disp = new DefaultRuleDispatcher(this, rules, null);
> GraphWalker ogw = new DefaultGraphWalker(disp);
> List<Node> topNodes = new ArrayList<>();
> topNodes.add(tree);
> ogw.startWalking(topNodes, null);
> Reporter: Zhou
> Priority: Major
>
> the sql is very long，about 3M size，use hive parse to parse the sql, it costs
> more than one hour to finish it. The reason is hive parse use
> IdentityHashMap, is not multi-thread safe, than it costs more than one hour
> to execute the code "while(true)"
>
> my code is :
> ParseDriver pd = new ParseDriver();
> ASTNode tree = pd.parse(query, context);
> while ((tree.getToken() == null) && (tree.getChildCount() > 0))
> { tree = (ASTNode) tree.getChild(0); }
> Map<Rule, NodeProcessor> rules = new LinkedHashMap<>();
> Dispatcher disp = new DefaultRuleDispatcher(this, rules, null);
> GraphWalker ogw = new DefaultGraphWalker(disp);
> List<Node> topNodes = new ArrayList<>();
> topNodes.add(tree);
> ogw.startWalking(topNodes, null);
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
