Daniel Dai created PIG-4314:
-------------------------------
Summary: BigData_5 hang on some machine
Key: PIG-4314
URL: https://issues.apache.org/jira/browse/PIG-4314
Project: Pig
Issue Type: Bug
Components: impl
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 0.14.0
Hang for both tez and mr. Here is the log:
{code}
2014-11-07 19:49:45,799 INFO [TezChild] relationalOperators.POPartialAgg:
Aggregated 10000 raw tuples. Processed tuples before aggregation = 0, after
aggregation = 676
2014-11-07 19:49:45,799 INFO [TezChild] relationalOperators.POPartialAgg: After
reduction, processed map: 676; raw map: 0
2014-11-07 19:49:45,800 INFO [TezChild] relationalOperators.POPartialAgg:
Observed reduction factor: from 10000 to 676 => 14.
2014-11-07 19:49:45,992 INFO [TezChild] relationalOperators.POPartialAgg:
Getting mem limits; considering 1 POPArtialAgg objects. with memory percentage
0.2
2014-11-07 19:49:46,290 INFO [TezChild] relationalOperators.POPartialAgg:
Estimated total tuples to buffer, based on 10000 tuples that took up 2364864
bytes: 1052574
2014-11-07 19:49:46,290 INFO [TezChild] relationalOperators.POPartialAgg:
Setting thresholds. Primary: 977390. Secondary: 75184
2014-11-07 19:49:50,641 INFO [Service Thread] util.SpillableMemoryManager:
first memory handler call - Collection threshold init = 216530944(211456K) used
= 724849000(707860K) committed = 927989760(906240K) max = 1146093568(1119232K)
2014-11-07 19:49:50,644 INFO [Service Thread] relationalOperators.POPartialAgg:
Spill triggered by SpillableMemoryManager
2014-11-07 19:49:50,993 INFO [TezChild] relationalOperators.POPartialAgg:
Aggregated 346545 raw tuples. Processed tuples before aggregation = 676, after
aggregation = 1352
2014-11-07 19:49:50,998 INFO [TezChild] relationalOperators.POPartialAgg:
Aggregated 1352 processed tuples to 676 tuples
2014-11-07 19:49:50,998 INFO [TezChild] relationalOperators.POPartialAgg:
Avoided emitting records during spill memory call.
2014-11-07 19:49:51,046 INFO [Service Thread] relationalOperators.POPartialAgg:
Finished spill for SpillableMemoryManager call
2014-11-07 19:49:51,851 INFO [Service Thread] util.SpillableMemoryManager:
Spilled an estimate of 81943920 bytes from 1 objects. init = 216530944(211456K)
used = 724849000(707860K) committed = 927989760(906240K) max =
1146093568(1119232K)
2014-11-07 19:49:59,644 INFO [Service Thread] relationalOperators.POPartialAgg:
Spill triggered by SpillableMemoryManager
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
