[jira] Commented: (PIG-1001) Generate more meaningful error message when one input file does not exist
[ https://issues.apache.org/jira/browse/PIG-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12776545#action_12776545 ] Nigel Daley commented on PIG-1001: -- Please ensure the issue is Assigned to the patch author when committing. Also, please provide a justification for why there is no unit test, then describe how you DID test it before you uploaded the patch. Generate more meaningful error message when one input file does not exist - Key: PIG-1001 URL: https://issues.apache.org/jira/browse/PIG-1001 Project: Pig Issue Type: Bug Affects Versions: 0.4.0 Reporter: Daniel Dai Fix For: 0.6.0 Attachments: PIG-1001-1.patch, PIG-1001-2.patch In the following query, if 1.txt does not exist, a = load '1.txt'; b = group a by $0; c = group b all; dump c; Pig throws error message ERROR 2100: file:/tmp/temp155054664/tmp1144108421 does not exist., Pig should deal with it with the error message Input file 1.txt not exist instead of those confusing messages. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1001) Generate more meaningful error message when one input file does not exist
[ https://issues.apache.org/jira/browse/PIG-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12776558#action_12776558 ] Daniel Dai commented on PIG-1001: - Hi, Nigel, Since this patch is all about error message and user experience, so it is hard to write a unit test case for that. I tested it manually with the following situations: 1. All jobs are successful 2. One complete failed job, and we stop launching dependent jobs 3. Two independent job, one fail, with and without -F option All of those give us desired error messages. Generate more meaningful error message when one input file does not exist - Key: PIG-1001 URL: https://issues.apache.org/jira/browse/PIG-1001 Project: Pig Issue Type: Bug Affects Versions: 0.4.0 Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.6.0 Attachments: PIG-1001-1.patch, PIG-1001-2.patch In the following query, if 1.txt does not exist, a = load '1.txt'; b = group a by $0; c = group b all; dump c; Pig throws error message ERROR 2100: file:/tmp/temp155054664/tmp1144108421 does not exist., Pig should deal with it with the error message Input file 1.txt not exist instead of those confusing messages. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1001) Generate more meaningful error message when one input file does not exist
[ https://issues.apache.org/jira/browse/PIG-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12771519#action_12771519 ] Hadoop QA commented on PIG-1001: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12423510/PIG-1001-2.patch against trunk revision 830757. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/31/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/31/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/31/console This message is automatically generated. Generate more meaningful error message when one input file does not exist - Key: PIG-1001 URL: https://issues.apache.org/jira/browse/PIG-1001 Project: Pig Issue Type: Bug Affects Versions: 0.4.0 Reporter: Daniel Dai Fix For: 0.6.0 Attachments: PIG-1001-1.patch, PIG-1001-2.patch In the following query, if 1.txt does not exist, a = load '1.txt'; b = group a by $0; c = group b all; dump c; Pig throws error message ERROR 2100: file:/tmp/temp155054664/tmp1144108421 does not exist., Pig should deal with it with the error message Input file 1.txt not exist instead of those confusing messages. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1001) Generate more meaningful error message when one input file does not exist
[
https://issues.apache.org/jira/browse/PIG-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12771072#action_12771072
]
Alan Gates commented on PIG-1001:
-
I have a question on this code in JobControlCompiler.updateMROpPlan:
{code}
for (Job job : completeFailedJobs) // remove all subsequent jobs
{
ListMapReduceOper fifo = new ArrayListMapReduceOper();
fifo.add(jobMroMap.get(job));
while (!fifo.isEmpty())
{
MapReduceOper mro = fifo.remove(0);
ListMapReduceOper succs = plan.getSuccessors(mro);
if (succs != null)
fifo.addAll(succs);
plan.remove(mro);
numRemoved++;
}
}
{code}
If we have a dependency graph like: A-B-C and A fails, won't the above code
only remove B and not C? OperatorPlan.getSuccessors() only gets immediate
successors, not all successors. I think you want OperatorPlan.trimBelow()
instead.
Generate more meaningful error message when one input file does not exist
-
Key: PIG-1001
URL: https://issues.apache.org/jira/browse/PIG-1001
Project: Pig
Issue Type: Bug
Affects Versions: 0.4.0
Reporter: Daniel Dai
Fix For: 0.6.0
Attachments: PIG-1001-1.patch
In the following query, if 1.txt does not exist,
a = load '1.txt';
b = group a by $0;
c = group b all;
dump c;
Pig throws error message ERROR 2100: file:/tmp/temp155054664/tmp1144108421
does not exist., Pig should deal with it with the error message Input file
1.txt not exist instead of those confusing messages.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1001) Generate more meaningful error message when one input file does not exist
[ https://issues.apache.org/jira/browse/PIG-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12765016#action_12765016 ] Hadoop QA commented on PIG-1001: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12421956/PIG-1001-1.patch against trunk revision 824446. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/21/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/21/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/21/console This message is automatically generated. Generate more meaningful error message when one input file does not exist - Key: PIG-1001 URL: https://issues.apache.org/jira/browse/PIG-1001 Project: Pig Issue Type: Bug Affects Versions: 0.4.0 Reporter: Daniel Dai Fix For: 0.6.0 Attachments: PIG-1001-1.patch In the following query, if 2.txt does not exist, a = load '1.txt'; b = order a by $0; c = load '2.txt'; d = order c by $0; e = join b by $0, d by $0; dump e; Pig throws error message ERROR 2100: file:/tmp/temp155054664/tmp1144108421 does not exist., Pig should deal with it with the error message Input file 2.txt not exist instead of those confusing messages. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
