[ 
https://issues.apache.org/jira/browse/PIG-250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates reassigned PIG-250:
------------------------------

    Assignee: Olga Natkovich

> Pig is broken with speculative execution
> ----------------------------------------
>
>                 Key: PIG-250
>                 URL: https://issues.apache.org/jira/browse/PIG-250
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Olga Natkovich
>            Assignee: Olga Natkovich
>             Fix For: 0.1.0
>
>         Attachments: PIG-250.patch, PIG-250_v2.patch
>
>
> If I have speculative execution turned on, the following script fails:
> a = load 'studenttab20m' as (name, age, gpa);
> b = load 'votertab10k' as (name, age, registration, contributions);
> c = filter a by age < '50';
> d = filter b by age < '50';
> e = cogroup c by (name, age), d by (name, age) parallel 10;
> f = foreach e generate flatten(c), flatten(d) parallel 10;
> g = group f by registration parallel 10;
> h = foreach g generate group, SUM(f.d::contributions) parallel 10;
> i = order h by ($1, $0);
> store i into 'out';
> I traced this to the fact that the first MR job produces one or more empty 
> outputs from the reducer. This happened on the reducers that happened to have 
> second task running.
> I am not sure what the issue is and I am working with hadoop guys to 
> investigate. Until this issue is resolved, I would like to trun speculative 
> execution off.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to