Re: how to control nested CROSS parallelism?

2014-01-21 Thread Serega Sheypak
ok, thanks! 2014/1/20 Pradeep Gollakota pradeep...@gmail.com It's strange that it's being executed on the Map-side. The group is a reduce side operation (I'm assuming) and it seems that the nested foreach would happen on Reduce-side after grouping. Have you looked at the MR plan to verify

how to control nested CROSS parallelism?

2014-01-20 Thread Serega Sheypak
Hi, I'm in trouble Here a part of code: itemGrp = GROUP itemProj1 BY sale_id PARALLEL 12; notFiltered = FOREACH itemGrp{ itemProj2 = FOREACH itemProj1 GENERATE FLATTEN( TOTUPLE(id, other_id)) as

Re: how to control nested CROSS parallelism?

2014-01-20 Thread Pradeep Gollakota
It's strange that it's being executed on the Map-side. The group is a reduce side operation (I'm assuming) and it seems that the nested foreach would happen on Reduce-side after grouping. Have you looked at the MR plan to verify that it is being executed Map-side? One thing to try might be to