[jira] Updated: (PIG-1025) Should be able to set job priority through Pig Latin
[ https://issues.apache.org/jira/browse/PIG-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Weil updated PIG-1025: Status: Patch Available (was: Open) Attaching updated patch. I'm still not sure how the last patch caused so many errors in MultiQueryLocal, but there was one spot where I would have effectively been calling PigContext.setProperty(jobPriority, null) if the priority was not set. I just added a null check before that call, and I no-op if the user never set job.priority. The patch now passes all tests for me when I run ant test-commit. Thanks Alan for manually applying the patch to test it. Should be able to set job priority through Pig Latin Key: PIG-1025 URL: https://issues.apache.org/jira/browse/PIG-1025 Project: Pig Issue Type: New Feature Components: grunt Affects Versions: 0.4.0 Reporter: Kevin Weil Priority: Minor Fix For: 0.6.0 Attachments: PIG-1025.patch, PIG-1025_2.patch Currently users can set the job name through Pig Latin by saying set job.name 'my job name' The ability to set the priority would also be nice, and the patch should be small. The goal is to be able to say set job.priority 'high' and throw a JobCreationException in the JobControlCompiler if the priority is not one of the allowed string values from the o.a.h.mapred.JobPriority enum: very_low, low, normal, high, very_high. Case insensitivity makes this a little nicer. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1025) Should be able to set job priority through Pig Latin
[ https://issues.apache.org/jira/browse/PIG-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Weil updated PIG-1025: Attachment: PIG-1025_2.patch Updated patch with the null check. Should be able to set job priority through Pig Latin Key: PIG-1025 URL: https://issues.apache.org/jira/browse/PIG-1025 Project: Pig Issue Type: New Feature Components: grunt Affects Versions: 0.4.0 Reporter: Kevin Weil Priority: Minor Fix For: 0.6.0 Attachments: PIG-1025.patch, PIG-1025_2.patch Currently users can set the job name through Pig Latin by saying set job.name 'my job name' The ability to set the priority would also be nice, and the patch should be small. The goal is to be able to say set job.priority 'high' and throw a JobCreationException in the JobControlCompiler if the priority is not one of the allowed string values from the o.a.h.mapred.JobPriority enum: very_low, low, normal, high, very_high. Case insensitivity makes this a little nicer. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1025) Should be able to set job priority through Pig Latin
[ https://issues.apache.org/jira/browse/PIG-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12767142#action_12767142 ] Kevin Weil commented on PIG-1025: - Looks like there was a cascading set of test failures unrelated to the patch. If I was somehow to blame here, let me know. Should be able to set job priority through Pig Latin Key: PIG-1025 URL: https://issues.apache.org/jira/browse/PIG-1025 Project: Pig Issue Type: New Feature Components: grunt Affects Versions: 0.4.0 Reporter: Kevin Weil Priority: Minor Fix For: 0.6.0 Attachments: PIG-1025.patch Currently users can set the job name through Pig Latin by saying set job.name 'my job name' The ability to set the priority would also be nice, and the patch should be small. The goal is to be able to say set job.priority 'high' and throw a JobCreationException in the JobControlCompiler if the priority is not one of the allowed string values from the o.a.h.mapred.JobPriority enum: very_low, low, normal, high, very_high. Case insensitivity makes this a little nicer. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (PIG-1025) Should be able to set job priority through Pig Latin
Should be able to set job priority through Pig Latin Key: PIG-1025 URL: https://issues.apache.org/jira/browse/PIG-1025 Project: Pig Issue Type: New Feature Components: grunt Affects Versions: 0.4.0 Reporter: Kevin Weil Priority: Minor Currently users can set the job name through Pig Latin by saying set job.name 'my job name' The ability to set the priority would also be nice, and the patch should be small. The goal is to be able to say set job.priority 'high' and throw a JobCreationException in the JobControlCompiler if the priority is not one of the allowed string values from the o.a.h.mapred.JobPriority enum: very_low, low, normal, high, very_high. Case insensitivity makes this a little nicer. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1025) Should be able to set job priority through Pig Latin
[ https://issues.apache.org/jira/browse/PIG-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Weil updated PIG-1025: Attachment: PIG-1025.patch Should be able to set job priority through Pig Latin Key: PIG-1025 URL: https://issues.apache.org/jira/browse/PIG-1025 Project: Pig Issue Type: New Feature Components: grunt Affects Versions: 0.4.0 Reporter: Kevin Weil Priority: Minor Fix For: 0.6.0 Attachments: PIG-1025.patch Currently users can set the job name through Pig Latin by saying set job.name 'my job name' The ability to set the priority would also be nice, and the patch should be small. The goal is to be able to say set job.priority 'high' and throw a JobCreationException in the JobControlCompiler if the priority is not one of the allowed string values from the o.a.h.mapred.JobPriority enum: very_low, low, normal, high, very_high. Case insensitivity makes this a little nicer. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1025) Should be able to set job priority through Pig Latin
[ https://issues.apache.org/jira/browse/PIG-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12766838#action_12766838 ] Kevin Weil commented on PIG-1025: - I very much agree that the test case is weak. I followed the model for the rest of the grunt tests, which are similarly weak :) Should be able to set job priority through Pig Latin Key: PIG-1025 URL: https://issues.apache.org/jira/browse/PIG-1025 Project: Pig Issue Type: New Feature Components: grunt Affects Versions: 0.4.0 Reporter: Kevin Weil Priority: Minor Fix For: 0.6.0 Attachments: PIG-1025.patch Currently users can set the job name through Pig Latin by saying set job.name 'my job name' The ability to set the priority would also be nice, and the patch should be small. The goal is to be able to say set job.priority 'high' and throw a JobCreationException in the JobControlCompiler if the priority is not one of the allowed string values from the o.a.h.mapred.JobPriority enum: very_low, low, normal, high, very_high. Case insensitivity makes this a little nicer. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-948) [Usability] Relating pig script with MR jobs
[ https://issues.apache.org/jira/browse/PIG-948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12759926#action_12759926 ] Kevin Weil commented on PIG-948: FWIW, I'd +1 Dmitriy's comment. Yes, it's a shame this isn't programmatically available via Hadoop, but come on. It's a single-line string concatenation. And it's FAR more convenient to print out the full url than to expect people to memorize the jobtracker url and sub in a parameter. One of these options is strictly correct, and the other has the overhead of a single line of code and is far more convenient to the end user :) [Usability] Relating pig script with MR jobs Key: PIG-948 URL: https://issues.apache.org/jira/browse/PIG-948 Project: Pig Issue Type: Improvement Components: impl Affects Versions: 0.4.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Minor Fix For: 0.6.0 Attachments: pig-948-2.patch, pig-948.patch Currently its hard to find a way to relate pig script with specific MR job. In a loaded cluster with multiple simultaneous job submissions, its not easy to figure out which specific MR jobs were launched for a given pig script. If Pig can provide this info, it will be useful to debug and monitor the jobs resulting from a pig script. At the very least, Pig should be able to provide user the following information 1) Job id of the launched job. 2) Complete web url of jobtracker running this job. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-573) Changes to make Pig run with Hadoop 19
[ https://issues.apache.org/jira/browse/PIG-573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12698910#action_12698910 ] Kevin Weil commented on PIG-573: What is the current status of this patch with pig 0.2? Since PIG-563 went in to 0.20, all that should be necessary is applying this single patch to the 0.20 release source, right? Changes to make Pig run with Hadoop 19 -- Key: PIG-573 URL: https://issues.apache.org/jira/browse/PIG-573 Project: Pig Issue Type: Task Affects Versions: 0.2.0 Reporter: Pradeep Kamath Assignee: Pradeep Kamath Attachments: hadoop19.jar, PIG-573-combinerflag.patch, PIG-573.patch This issue tracks changes to Pig code to make it work with Hadoop-0.19.x -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-569) Inconsistency with Hadoop in Pig load statements involving globs with subdirectories
[
https://issues.apache.org/jira/browse/PIG-569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Weil updated PIG-569:
---
Environment: FC Linux x86/64, Pig revision 724576 (was: FC Linux x86/64)
Inconsistency with Hadoop in Pig load statements involving globs with
subdirectories
Key: PIG-569
URL: https://issues.apache.org/jira/browse/PIG-569
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: types_branch
Environment: FC Linux x86/64, Pig revision 724576
Reporter: Kevin Weil
Fix For: types_branch
Pig cannot handle LOAD statements with Hadoop globs where the globs have
subdirectories. For example,
A = LOAD 'dir/{dir1/subdir1,dir2/subdir2,dir3/subdir3}' USING ...
A similar statement in Hadoop, hadoop dfs -ls
dir/{dir1/subdir1,dir2/subdir2,dir3/subdir3}, does work correctly.
The output of running the above load statement in pig, built from svn
revision 724576, is:
2008-12-17 12:02:28,480 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete
2008-12-17 12:02:28,480 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Map reduce job failed
2008-12-17 12:02:28,480 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- java.io.IOException: Unable to get collect for pattern
dir/{dir1/subdir1,dir2/subdir2,dir3/subdir3}} [Failed to obtain glob for
dir/{dir1/subdir1,dir2/subdir2,dir3/subdir3}]
at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.asCollection(HDataStorage.java:231)
at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.asCollection(HDataStorage.java:40)
at
org.apache.pig.impl.io.FileLocalizer.globMatchesFiles(FileLocalizer.java:486)
at
org.apache.pig.impl.io.FileLocalizer.fileExists(FileLocalizer.java:455)
at
org.apache.pig.backend.executionengine.PigSlicer.validate(PigSlicer.java:108)
at
org.apache.pig.impl.io.ValidatingInputFileSpec.validate(ValidatingInputFileSpec.java:59)
at
org.apache.pig.impl.io.ValidatingInputFileSpec.init(ValidatingInputFileSpec.java:44)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:200)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:742)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:370)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.pig.backend.datastorage.DataStorageException: Failed to
obtain glob for dir/{dir1/subdir1,dir2/subdir2,dir3/subdir3}
... 13 more
Caused by: java.io.IOException: Illegal file pattern: Expecting set closure
character or end of range, or } for glob {dir1 at 5
at
org.apache.hadoop.fs.FileSystem$GlobFilter.error(FileSystem.java:1084)
at
org.apache.hadoop.fs.FileSystem$GlobFilter.setRegex(FileSystem.java:1069)
at
org.apache.hadoop.fs.FileSystem$GlobFilter.init(FileSystem.java:987)
at org.apache.hadoop.fs.FileSystem.globPathsLevel(FileSystem.java:953)
at org.apache.hadoop.fs.FileSystem.globPathsLevel(FileSystem.java:962)
at org.apache.hadoop.fs.FileSystem.globPathsLevel(FileSystem.java:962)
at org.apache.hadoop.fs.FileSystem.globPathsLevel(FileSystem.java:962)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:902)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:862)
at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.asCollection(HDataStorage.java:215)
... 12 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
