Hi
I launched the attached pig job on pig-12 with hadoop MRv1 with the
attached data, but the FILTER function causes the job to get stuck in an
infinite loop.
pig -p pPeriod=201312 -f test.pig
The thread in question seems to be stuck forever inside while loop of
runPipeline method.
stack trace:
-----------
"main" prio=10 tid=0x00007fd74800b000 nid=0x2f63 runnable
[0x00007fd750d50000]
java.lang.Thread.State: RUNNABLE
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.
relationalOperators.POForEach.getNextTuple(POForEach.java:217)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
PigGenericMapBase.map(PigGenericMapBase.java:277)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:680)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:346)
at org.apache.hadoop.mapred.Child$4.run(Child.java:282)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(
UserGroupInformation.java:1117)
at org.apache.hadoop.mapred.Child.main(Child.java:271)
org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/
PigGenericMapBase.java:
protected void *runPipeline*(PhysicalOperator leaf) throws IOException,
InterruptedException {
while(true){
Result res = leaf.getNext(DUMMYTUPLE);
if(res.returnStatus==POStatus.STATUS_OK){
collect(outputCollector,(Tuple)res.result);
continue;
}
....
Whats the suggested code fix here?
Thanks,
Suhas.
20100101|1 WEEK ENDING 01/02/10|2010-Jan-02|201001|1468
20100102|1 WEEK ENDING 01/09/10|2010-Jan-09|201001|1469
20100103|1 WEEK ENDING 01/16/10|2010-Jan-16|201001|1470
20100104|1 WEEK ENDING 01/23/10|2010-Jan-23|201001|1471
20100205|1 WEEK ENDING 01/30/10|2010-Jan-30|201002|1472
20100206|1 WEEK ENDING 02/06/10|2010-Feb-06|201002|1473
20100207|1 WEEK ENDING 02/13/10|2010-Feb-13|201002|1474
20100208|1 WEEK ENDING 02/20/10|2010-Feb-20|201002|1475
20100309|1 WEEK ENDING 02/27/10|2010-Feb-27|201003|1476
20100310|1 WEEK ENDING 03/06/10|2010-Mar-06|201003|1477
20100311|1 WEEK ENDING 03/13/10|2010-Mar-13|201003|1478
20100312|1 WEEK ENDING 03/20/10|2010-Mar-20|201003|1479
20100413|1 WEEK ENDING 03/27/10|2010-Mar-27|201004|1480
20100414|1 WEEK ENDING 04/03/10|2010-Apr-03|201004|1481
20100415|1 WEEK ENDING 04/10/10|2010-Apr-10|201004|1482
20100416|1 WEEK ENDING 04/17/10|2010-Apr-17|201004|1483
20100517|1 WEEK ENDING 04/24/10|2010-Apr-24|201005|1484
20100518|1 WEEK ENDING 05/01/10|2010-May-01|201005|1485
20100519|1 WEEK ENDING 05/08/10|2010-May-08|201005|1486
20100520|1 WEEK ENDING 05/15/10|2010-May-15|201005|1487
20100621|1 WEEK ENDING 05/22/10|2010-May-22|201006|1488
20100622|1 WEEK ENDING 05/29/10|2010-May-29|201006|1489
20100623|1 WEEK ENDING 06/05/10|2010-Jun-05|201006|1490
20100624|1 WEEK ENDING 06/12/10|2010-Jun-12|201006|1491
20100725|1 WEEK ENDING 06/19/10|2010-Jun-19|201007|1492
20100726|1 WEEK ENDING 06/26/10|2010-Jun-26|201007|1493
20100727|1 WEEK ENDING 07/03/10|2010-Jul-03|201007|1494
20100728|1 WEEK ENDING 07/10/10|2010-Jul-10|201007|1495
20100829|1 WEEK ENDING 07/17/10|2010-Jul-17|201008|1496
20100830|1 WEEK ENDING 07/24/10|2010-Jul-24|201008|1497
20100831|1 WEEK ENDING 07/31/10|2010-Jul-31|201008|1498
20100832|1 WEEK ENDING 08/07/10|2010-Aug-07|201008|1499
20100933|1 WEEK ENDING 08/14/10|2010-Aug-14|201009|1500
20100934|1 WEEK ENDING 08/21/10|2010-Aug-21|201009|1501
20100935|1 WEEK ENDING 08/28/10|2010-Aug-28|201009|1502
20100936|1 WEEK ENDING 09/04/10|2010-Sep-04|201009|1503
20101037|1 WEEK ENDING 09/11/10|2010-Sep-11|201010|1504
20101038|1 WEEK ENDING 09/18/10|2010-Sep-18|201010|1505
20101039|1 WEEK ENDING 09/25/10|2010-Sep-25|201010|1506
20101040|1 WEEK ENDING 10/02/10|2010-Oct-02|201010|1507
20101141|1 WEEK ENDING 10/09/10|2010-Oct-09|201011|1508
20101142|1 WEEK ENDING 10/16/10|2010-Oct-16|201011|1509
20101143|1 WEEK ENDING 10/23/10|2010-Oct-23|201011|1510
20101144|1 WEEK ENDING 10/30/10|2010-Oct-30|201011|1511
20101245|1 WEEK ENDING 11/06/10|2010-Nov-06|201012|1512
20101246|1 WEEK ENDING 11/13/10|2010-Nov-13|201012|1513
20101247|1 WEEK ENDING 11/20/10|2010-Nov-20|201012|1514
20101248|1 WEEK ENDING 11/27/10|2010-Nov-27|201012|1515
20101349|1 WEEK ENDING 12/04/10|2010-Dec-04|201013|1516
20101350|1 WEEK ENDING 12/11/10|2010-Dec-11|201013|1517
20101351|1 WEEK ENDING 12/18/10|2010-Dec-18|201013|1518
20101352|1 WEEK ENDING 12/25/10|2010-Dec-25|201013|1519
20110101|1 WEEK ENDING 01/01/11|2011-Jan-01|201101|1520
20110102|1 WEEK ENDING 01/08/11|2011-Jan-08|201101|1521
20110103|1 WEEK ENDING 01/15/11|2011-Jan-15|201101|1522
20110104|1 WEEK ENDING 01/22/11|2011-Jan-22|201101|1523
20110205|1 WEEK ENDING 01/29/11|2011-Jan-29|201102|1524
20110206|1 WEEK ENDING 02/05/11|2011-Feb-05|201102|1525
20110207|1 WEEK ENDING 02/12/11|2011-Feb-12|201102|1526
20110208|1 WEEK ENDING 02/19/11|2011-Feb-19|201102|1527
20110309|1 WEEK ENDING 02/26/11|2011-Feb-26|201103|1528
20110310|1 WEEK ENDING 03/05/11|2011-Mar-05|201103|1529
20110311|1 WEEK ENDING 03/12/11|2011-Mar-12|201103|1530
20110312|1 WEEK ENDING 03/19/11|2011-Mar-19|201103|1531
20110413|1 WEEK ENDING 03/26/11|2011-Mar-26|201104|1532
20110414|1 WEEK ENDING 04/02/11|2011-Apr-02|201104|1533
20110415|1 WEEK ENDING 04/09/11|2011-Apr-09|201104|1534
20110416|1 WEEK ENDING 04/16/11|2011-Apr-16|201104|1535
20110517|1 WEEK ENDING 04/23/11|2011-Apr-23|201105|1536
20110518|1 WEEK ENDING 04/30/11|2011-Apr-30|201105|1537
20110519|1 WEEK ENDING 05/07/11|2011-May-07|201105|1538
20110520|1 WEEK ENDING 05/14/11|2011-May-14|201105|1539
20110621|1 WEEK ENDING 05/21/11|2011-May-21|201106|1540
20110622|1 WEEK ENDING 05/28/11|2011-May-28|201106|1541
20110623|1 WEEK ENDING 06/04/11|2011-Jun-04|201106|1542
20110624|1 WEEK ENDING 06/11/11|2011-Jun-11|201106|1543
20110725|1 WEEK ENDING 06/18/11|2011-Jun-18|201107|1544
20110726|1 WEEK ENDING 06/25/11|2011-Jun-25|201107|1545
20110727|1 WEEK ENDING 07/02/11|2011-Jul-02|201107|1546
20110728|1 WEEK ENDING 07/09/11|2011-Jul-09|201107|1547
20110829|1 WEEK ENDING 07/16/11|2011-Jul-16|201108|1548
20110830|1 WEEK ENDING 07/23/11|2011-Jul-23|201108|1549
20110831|1 WEEK ENDING 07/30/11|2011-Jul-30|201108|1550
20110832|1 WEEK ENDING 08/06/11|2011-Aug-06|201108|1551
20110933|1 WEEK ENDING 08/13/11|2011-Aug-13|201109|1552
20110934|1 WEEK ENDING 08/20/11|2011-Aug-20|201109|1553
20110935|1 WEEK ENDING 08/27/11|2011-Aug-27|201109|1554
20110936|1 WEEK ENDING 09/03/11|2011-Sep-03|201109|1555
20111037|1 WEEK ENDING 09/10/11|2011-Sep-10|201110|1556
20111038|1 WEEK ENDING 09/17/11|2011-Sep-17|201110|1557
20111039|1 WEEK ENDING 09/24/11|2011-Sep-24|201110|1558
20111040|1 WEEK ENDING 10/01/11|2011-Oct-01|201110|1559
20111141|1 WEEK ENDING 10/08/11|2011-Oct-08|201111|1560
20111142|1 WEEK ENDING 10/15/11|2011-Oct-15|201111|1561
20111143|1 WEEK ENDING 10/22/11|2011-Oct-22|201111|1562
20111144|1 WEEK ENDING 10/29/11|2011-Oct-29|201111|1563
20111245|1 WEEK ENDING 11/05/11|2011-Nov-05|201112|1564
20111246|1 WEEK ENDING 11/12/11|2011-Nov-12|201112|1565
20111247|1 WEEK ENDING 11/19/11|2011-Nov-19|201112|1566
20111248|1 WEEK ENDING 11/26/11|2011-Nov-26|201112|1567
20111349|1 WEEK ENDING 12/03/11|2011-Dec-03|201113|1568
20111350|1 WEEK ENDING 12/10/11|2011-Dec-10|201113|1569
20111351|1 WEEK ENDING 12/17/11|2011-Dec-17|201113|1570
20111352|1 WEEK ENDING 12/24/11|2011-Dec-24|201113|1571
20120101|1 WEEK ENDING 12/31/11|2011-Dec-31|201201|1572
20120102|1 WEEK ENDING 01/07/12|2012-Jan-07|201201|1573
20120103|1 WEEK ENDING 01/14/12|2012-Jan-14|201201|1574
20120104|1 WEEK ENDING 01/21/12|2012-Jan-21|201201|1575
20120205|1 WEEK ENDING 01/28/12|2012-Jan-28|201202|1576
20120206|1 WEEK ENDING 02/04/12|2012-Feb-04|201202|1577
20120207|1 WEEK ENDING 02/11/12|2012-Feb-11|201202|1578
20120208|1 WEEK ENDING 02/18/12|2012-Feb-18|201202|1579
20120309|1 WEEK ENDING 02/25/12|2012-Feb-25|201203|1580
20120310|1 WEEK ENDING 03/03/12|2012-Mar-03|201203|1581
20120311|1 WEEK ENDING 03/10/12|2012-Mar-10|201203|1582
20120312|1 WEEK ENDING 03/17/12|2012-Mar-17|201203|1583
20120413|1 WEEK ENDING 03/24/12|2012-Mar-24|201204|1584
20120414|1 WEEK ENDING 03/31/12|2012-Mar-31|201204|1585
20120415|1 WEEK ENDING 04/07/12|2012-Apr-07|201204|1586
20120416|1 WEEK ENDING 04/14/12|2012-Apr-14|201204|1587
20120517|1 WEEK ENDING 04/21/12|2012-Apr-21|201205|1588
20120518|1 WEEK ENDING 04/28/12|2012-Apr-28|201205|1589
20120519|1 WEEK ENDING 05/05/12|2012-May-05|201205|1590
20120520|1 WEEK ENDING 05/12/12|2012-May-12|201205|1591
20120621|1 WEEK ENDING 05/19/12|2012-May-19|201206|1592
20120622|1 WEEK ENDING 05/26/12|2012-May-26|201206|1593
20120623|1 WEEK ENDING 06/02/12|2012-Jun-02|201206|1594
20120624|1 WEEK ENDING 06/09/12|2012-Jun-09|201206|1595
20120725|1 WEEK ENDING 06/16/12|2012-Jun-16|201207|1596
20120726|1 WEEK ENDING 06/23/12|2012-Jun-23|201207|1597
20120727|1 WEEK ENDING 06/30/12|2012-Jun-30|201207|1598
20120728|1 WEEK ENDING 07/07/12|2012-Jul-07|201207|1599
20120829|1 WEEK ENDING 07/14/12|2012-Jul-14|201208|1600
20120830|1 WEEK ENDING 07/21/12|2012-Jul-21|201208|1601
20120831|1 WEEK ENDING 07/28/12|2012-Jul-28|201208|1602
20120832|1 WEEK ENDING 08/04/12|2012-Aug-04|201208|1603
20120933|1 WEEK ENDING 08/11/12|2012-Aug-11|201209|1604
20120934|1 WEEK ENDING 08/18/12|2012-Aug-18|201209|1605
20120935|1 WEEK ENDING 08/25/12|2012-Aug-25|201209|1606
20120936|1 WEEK ENDING 09/01/12|2012-Sep-01|201209|1607
20121037|1 WEEK ENDING 09/08/12|2012-Sep-08|201210|1608
20121038|1 WEEK ENDING 09/15/12|2012-Sep-15|201210|1609
20121039|1 WEEK ENDING 09/22/12|2012-Sep-22|201210|1610
20121040|1 WEEK ENDING 09/29/12|2012-Sep-29|201210|1611
20121141|1 WEEK ENDING 10/06/12|2012-Oct-06|201211|1612
20121142|1 WEEK ENDING 10/13/12|2012-Oct-13|201211|1613
20121143|1 WEEK ENDING 10/20/12|2012-Oct-20|201211|1614
20121144|1 WEEK ENDING 10/27/12|2012-Oct-27|201211|1615
20121245|1 WEEK ENDING 11/03/12|2012-Nov-03|201212|1616
20121246|1 WEEK ENDING 11/10/12|2012-Nov-10|201212|1617
20121247|1 WEEK ENDING 11/17/12|2012-Nov-17|201212|1618
20121248|1 WEEK ENDING 11/24/12|2012-Nov-24|201212|1619
20121349|1 WEEK ENDING 12/01/12|2012-Dec-01|201213|1620
20121350|1 WEEK ENDING 12/08/12|2012-Dec-08|201213|1621
20121351|1 WEEK ENDING 12/15/12|2012-Dec-15|201213|1622
20121352|1 WEEK ENDING 12/22/12|2012-Dec-22|201213|1623
20130101|1 WEEK ENDING 12/29/12|2012-Dec-29|201301|1624
20130102|1 WEEK ENDING 01/05/13|2013-Jan-05|201301|1625
20130103|1 WEEK ENDING 01/12/13|2013-Jan-12|201301|1626
20130104|1 WEEK ENDING 01/19/13|2013-Jan-19|201301|1627
20130205|1 WEEK ENDING 01/26/13|2013-Jan-26|201302|1628
20130206|1 WEEK ENDING 02/02/13|2013-Feb-02|201302|1629
20130207|1 WEEK ENDING 02/09/13|2013-Feb-09|201302|1630
20130208|1 WEEK ENDING 02/16/13|2013-Feb-16|201302|1631
20130309|1 WEEK ENDING 02/23/13|2013-Feb-23|201303|1632
20130310|1 WEEK ENDING 03/02/13|2013-Mar-02|201303|1633
20130311|1 WEEK ENDING 03/09/13|2013-Mar-09|201303|1634
20130312|1 WEEK ENDING 03/16/13|2013-Mar-16|201303|1635
20130413|1 WEEK ENDING 03/23/13|2013-Mar-23|201304|1636
20130414|1 WEEK ENDING 03/30/13|2013-Mar-30|201304|1637
20130415|1 WEEK ENDING 04/06/13|2013-Apr-06|201304|1638
20130416|1 WEEK ENDING 04/13/13|2013-Apr-13|201304|1639
20130517|1 WEEK ENDING 04/20/13|2013-Apr-20|201305|1640
20130518|1 WEEK ENDING 04/27/13|2013-Apr-27|201305|1641
20130519|1 WEEK ENDING 05/04/13|2013-May-04|201305|1642
20130520|1 WEEK ENDING 05/11/13|2013-May-11|201305|1643
20130621|1 WEEK ENDING 05/18/13|2013-May-18|201306|1644
20130622|1 WEEK ENDING 05/25/13|2013-May-25|201306|1645
20130623|1 WEEK ENDING 06/01/13|2013-Jun-01|201306|1646
20130624|1 WEEK ENDING 06/08/13|2013-Jun-08|201306|1647
20130725|1 WEEK ENDING 06/15/13|2013-Jun-15|201307|1648
20130726|1 WEEK ENDING 06/22/13|2013-Jun-22|201307|1649
20130727|1 WEEK ENDING 06/29/13|2013-Jun-29|201307|1650
20130728|1 WEEK ENDING 07/06/13|2013-Jul-06|201307|1651
20130829|1 WEEK ENDING 07/13/13|2013-Jul-13|201308|1652
20130830|1 WEEK ENDING 07/20/13|2013-Jul-20|201308|1653
20130831|1 WEEK ENDING 07/27/13|2013-Jul-27|201308|1654
20130832|1 WEEK ENDING 08/03/13|2013-Aug-03|201308|1655
20130933|1 WEEK ENDING 08/10/13|2013-Aug-10|201309|1656
20130934|1 WEEK ENDING 08/17/13|2013-Aug-17|201309|1657
20130935|1 WEEK ENDING 08/24/13|2013-Aug-24|201309|1658
20130936|1 WEEK ENDING 08/31/13|2013-Aug-31|201309|1659
20131037|1 WEEK ENDING 09/07/13|2013-Sep-07|201310|1660
20131038|1 WEEK ENDING 09/14/13|2013-Sep-14|201310|1661
20131039|1 WEEK ENDING 09/21/13|2013-Sep-21|201310|1662
20131040|1 WEEK ENDING 09/28/13|2013-Sep-28|201310|1663
20131141|1 WEEK ENDING 10/05/13|2013-Oct-05|201311|1664
20131142|1 WEEK ENDING 10/12/13|2013-Oct-12|201311|1665
20131143|1 WEEK ENDING 10/19/13|2013-Oct-19|201311|1666
20131144|1 WEEK ENDING 10/26/13|2013-Oct-26|201311|1667
20131245|1 WEEK ENDING 11/02/13|2013-Nov-02|201312|1668
20131246|1 WEEK ENDING 11/09/13|2013-Nov-09|201312|1669
20131247|1 WEEK ENDING 11/16/13|2013-Nov-16|201312|1670
20131248|1 WEEK ENDING 11/23/13|2013-Nov-23|201312|1671
20131349|1 WEEK ENDING 11/30/13|2013-Nov-30|201313|1672
20131350|1 WEEK ENDING 12/07/13|2013-Dec-07|201313|1673
20131351|1 WEEK ENDING 12/14/13|2013-Dec-14|201313|1674
20131352|1 WEEK ENDING 12/21/13|2013-Dec-21|201313|1675
