[jira] Created: (PIG-1657) reduce the ivy verbosity during build.
reduce the ivy verbosity during build. -- Key: PIG-1657 URL: https://issues.apache.org/jira/browse/PIG-1657 Project: Pig Issue Type: Improvement Reporter: Giridharan Kesavan ivy is very verbose while doing build, making it less verbose would let us see what the builds actually does.. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1607) pig should have separate javadoc.jar in the maven repository
[ https://issues.apache.org/jira/browse/PIG-1607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12916594#action_12916594 ] Giridharan Kesavan commented on PIG-1607: - looks good +1 able to do mvn-install and mvn-deploy to install/deploy javadoc jar to the fs and apache mvn repo. pig should have separate javadoc.jar in the maven repository Key: PIG-1607 URL: https://issues.apache.org/jira/browse/PIG-1607 Project: Pig Issue Type: Bug Reporter: niraj rai Assignee: niraj rai Attachments: PIG-1607_0.patch, PIG-1607_1.patch, PIG-1607_2.patch, PIG-1607_3.patch, PIG-1607_4.patch At this moment, javadoc is part of the source.jar but pig should have separate javadoc.jar in the maven repository. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1608) pig should always include pig-default.properties and pig.properties in the pig.jar
[ https://issues.apache.org/jira/browse/PIG-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1608: Status: Patch Available (was: Open) pig should always include pig-default.properties and pig.properties in the pig.jar -- Key: PIG-1608 URL: https://issues.apache.org/jira/browse/PIG-1608 Project: Pig Issue Type: Bug Reporter: niraj rai Assignee: niraj rai Attachments: PIG-1608_0.patch pig should always include pig-default.properties and pig.properties as a part of the pig.jar file -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1608) pig should always include pig-default.properties and pig.properties in the pig.jar
[ https://issues.apache.org/jira/browse/PIG-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1608: Status: Open (was: Patch Available) re-submiting patch to hudson .. pig should always include pig-default.properties and pig.properties in the pig.jar -- Key: PIG-1608 URL: https://issues.apache.org/jira/browse/PIG-1608 Project: Pig Issue Type: Bug Reporter: niraj rai Assignee: niraj rai Attachments: PIG-1608_0.patch pig should always include pig-default.properties and pig.properties as a part of the pig.jar file -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1608) pig should always include pig-default.properties and pig.properties in the pig.jar
[ https://issues.apache.org/jira/browse/PIG-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1608: Status: Open (was: Patch Available) pig should always include pig-default.properties and pig.properties in the pig.jar -- Key: PIG-1608 URL: https://issues.apache.org/jira/browse/PIG-1608 Project: Pig Issue Type: Bug Reporter: niraj rai Assignee: niraj rai Attachments: PIG-1608_0.patch pig should always include pig-default.properties and pig.properties as a part of the pig.jar file -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1608) pig should always include pig-default.properties and pig.properties in the pig.jar
[ https://issues.apache.org/jira/browse/PIG-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1608: Status: Patch Available (was: Open) pig should always include pig-default.properties and pig.properties in the pig.jar -- Key: PIG-1608 URL: https://issues.apache.org/jira/browse/PIG-1608 Project: Pig Issue Type: Bug Reporter: niraj rai Assignee: niraj rai Attachments: PIG-1608_0.patch pig should always include pig-default.properties and pig.properties as a part of the pig.jar file -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1562) Fix the version for the dependent packages for the maven
[ https://issues.apache.org/jira/browse/PIG-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1290#action_1290 ] Giridharan Kesavan commented on PIG-1562: - I verified this patch by uploading artifacts to the staging repo by using my gpg keys. This works fine.. But, It publishes artifacts to org/apache/hadoop/pig-core/jar and pom artifacts. i guess it should be pig and not pig-core Fix the version for the dependent packages for the maven - Key: PIG-1562 URL: https://issues.apache.org/jira/browse/PIG-1562 Project: Pig Issue Type: Bug Reporter: niraj rai Assignee: niraj rai Fix For: 0.8.0 Attachments: PIG-1562_1.patch, PIG-1562_2.patch, PIG_1562_0.patch We need to fix the set version so that, version is properly set for the dependent packages in the maven repository. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1562) Fix the version for the dependent packages for the maven
[ https://issues.apache.org/jira/browse/PIG-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12907791#action_12907791 ] Giridharan Kesavan commented on PIG-1562: - It looks like the latest patch publishes as org/apache/hadoop/pig/jar's. it looks good except for javadoc artifact as mentioned in pig-1334. Fix the version for the dependent packages for the maven - Key: PIG-1562 URL: https://issues.apache.org/jira/browse/PIG-1562 Project: Pig Issue Type: Bug Reporter: niraj rai Assignee: niraj rai Fix For: 0.8.0 Attachments: PIG-1562_1.patch, PIG-1562_2.patch, PIG_1562_0.patch We need to fix the set version so that, version is properly set for the dependent packages in the maven repository. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (PIG-1582) upgrade commons-logging version with ivy
upgrade commons-logging version with ivy Key: PIG-1582 URL: https://issues.apache.org/jira/browse/PIG-1582 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan to upgrade the commons-logging version for pig from 1.0.3 to 1.1.1 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1582) upgrade commons-logging version with ivy
[ https://issues.apache.org/jira/browse/PIG-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1582: Attachment: pig-1582.patch upgrade commons-logging version with ivy Key: PIG-1582 URL: https://issues.apache.org/jira/browse/PIG-1582 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-1582.patch to upgrade the commons-logging version for pig from 1.0.3 to 1.1.1 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1582) upgrade commons-logging version with ivy
[ https://issues.apache.org/jira/browse/PIG-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1582: Status: Patch Available (was: Open) upgrade commons-logging version with ivy Key: PIG-1582 URL: https://issues.apache.org/jira/browse/PIG-1582 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-1582.patch to upgrade the commons-logging version for pig from 1.0.3 to 1.1.1 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1582) upgrade commons-logging version with ivy
[ https://issues.apache.org/jira/browse/PIG-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1582: Status: Resolved (was: Patch Available) Fix Version/s: 0.8.0 Resolution: Fixed upgrade commons-logging version with ivy Key: PIG-1582 URL: https://issues.apache.org/jira/browse/PIG-1582 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Fix For: 0.8.0 Attachments: pig-1582.patch to upgrade the commons-logging version for pig from 1.0.3 to 1.1.1 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1583) piggybank unit test TestLookupInFiles is broken
[ https://issues.apache.org/jira/browse/PIG-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1583: Status: Open (was: Patch Available) submitting to hudson piggybank unit test TestLookupInFiles is broken --- Key: PIG-1583 URL: https://issues.apache.org/jira/browse/PIG-1583 Project: Pig Issue Type: Bug Components: impl Affects Versions: 0.8.0 Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.8.0 Attachments: PIG-1583-1.patch Error message: 10/08/31 09:32:12 INFO mapred.TaskInProgress: Error from attempt_20100831093139211_0001_m_00_3: org.apache.pig.backend.executionengine.ExecException: ERROR 2078: Caught error from UDF: org.apache.pig.piggybank.evaluation.string.LookupInFiles [LookupInFiles : Cannot open file one] at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:262) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:283) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:355) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.io.IOException: LookupInFiles : Cannot open file one at org.apache.pig.piggybank.evaluation.string.LookupInFiles.init(LookupInFiles.java:92) at org.apache.pig.piggybank.evaluation.string.LookupInFiles.exec(LookupInFiles.java:115) at org.apache.pig.piggybank.evaluation.string.LookupInFiles.exec(LookupInFiles.java:49) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:229) ... 10 more Caused by: java.io.IOException: hdfs://localhost:47453/user/hadoopqa/one does not exist at org.apache.pig.impl.io.FileLocalizer.openDFSFile(FileLocalizer.java:224) at org.apache.pig.impl.io.FileLocalizer.openDFSFile(FileLocalizer.java:172) at org.apache.pig.piggybank.evaluation.string.LookupInFiles.init(LookupInFiles.java:89) ... 13 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1583) piggybank unit test TestLookupInFiles is broken
[ https://issues.apache.org/jira/browse/PIG-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1583: Status: Patch Available (was: Open) piggybank unit test TestLookupInFiles is broken --- Key: PIG-1583 URL: https://issues.apache.org/jira/browse/PIG-1583 Project: Pig Issue Type: Bug Components: impl Affects Versions: 0.8.0 Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.8.0 Attachments: PIG-1583-1.patch Error message: 10/08/31 09:32:12 INFO mapred.TaskInProgress: Error from attempt_20100831093139211_0001_m_00_3: org.apache.pig.backend.executionengine.ExecException: ERROR 2078: Caught error from UDF: org.apache.pig.piggybank.evaluation.string.LookupInFiles [LookupInFiles : Cannot open file one] at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:262) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:283) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:355) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.io.IOException: LookupInFiles : Cannot open file one at org.apache.pig.piggybank.evaluation.string.LookupInFiles.init(LookupInFiles.java:92) at org.apache.pig.piggybank.evaluation.string.LookupInFiles.exec(LookupInFiles.java:115) at org.apache.pig.piggybank.evaluation.string.LookupInFiles.exec(LookupInFiles.java:49) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:229) ... 10 more Caused by: java.io.IOException: hdfs://localhost:47453/user/hadoopqa/one does not exist at org.apache.pig.impl.io.FileLocalizer.openDFSFile(FileLocalizer.java:224) at org.apache.pig.impl.io.FileLocalizer.openDFSFile(FileLocalizer.java:172) at org.apache.pig.piggybank.evaluation.string.LookupInFiles.init(LookupInFiles.java:89) ... 13 more -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1560) Build target 'checkstyle' fails
[
https://issues.apache.org/jira/browse/PIG-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giridharan Kesavan updated PIG-1560:
Attachment: pig-1560.patch
This patch fixes the checkstyle target build failure.
Build target 'checkstyle' fails
---
Key: PIG-1560
URL: https://issues.apache.org/jira/browse/PIG-1560
Project: Pig
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Richard Ding
Assignee: Giridharan Kesavan
Fix For: 0.8.0
Attachments: pig-1560.patch
Stack trace:
{code}
/trunk/build.xml:894: java.lang.NoClassDefFoundError:
org/apache/commons/logging/LogFactory
at
org.apache.commons.beanutils.ConvertUtilsBean.init(ConvertUtilsBean.java:130)
at
com.puppycrawl.tools.checkstyle.api.AutomaticBean.createBeanUtilsBean(AutomaticBean.java:73)
at
com.puppycrawl.tools.checkstyle.api.AutomaticBean.contextualize(AutomaticBean.java:222)
at
com.puppycrawl.tools.checkstyle.CheckStyleTask.createChecker(CheckStyleTask.java:372)
at
com.puppycrawl.tools.checkstyle.CheckStyleTask.realExecute(CheckStyleTask.java:304)
at
com.puppycrawl.tools.checkstyle.CheckStyleTask.execute(CheckStyleTask.java:265)
at
org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at
org.apache.tools.ant.Project.executeSortedTargets(Project.java:1360)
at org.apache.tools.ant.Project.executeTarget(Project.java:1329)
at
org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1212)
at org.apache.tools.ant.Main.runBuild(Main.java:801)
at org.apache.tools.ant.Main.startAnt(Main.java:218)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
Caused by: java.lang.ClassNotFoundException:
org.apache.commons.logging.LogFactory
at
org.apache.tools.ant.AntClassLoader.findClassInComponents(AntClassLoader.java:1386)
at
org.apache.tools.ant.AntClassLoader.findClass(AntClassLoader.java:1336)
at
org.apache.tools.ant.AntClassLoader.loadClass(AntClassLoader.java:1074)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 22 more
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1452) to remove hadoop20.jar from lib and use hadoop from the apache maven repo.
[ https://issues.apache.org/jira/browse/PIG-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894651#action_12894651 ] Giridharan Kesavan commented on PIG-1452: - we can just include the list of jars that you may want it as part of the pig jar. Only reason I didnt do it in first place is that I didnt want to write 15 to 20 lines of zipfileset. Looks like its necessary for the given scenario. As long as someone can give me the list of jars that should go into the pig.jar, I shall add it to the pig.jar to remove hadoop20.jar from lib and use hadoop from the apache maven repo. -- Key: PIG-1452 URL: https://issues.apache.org/jira/browse/PIG-1452 Project: Pig Issue Type: Improvement Components: build Affects Versions: 0.8.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Fix For: 0.8.0 Attachments: PIG-1452.PATCH, PIG-1452V2.PATCH pig use ivy for dependency management. But still it uses hadoop20.jar from the lib folder. Now that we have the hadoop-0.20.2 artifacts available in the maven repo, pig should leverage ivy for resolving/retrieving hadoop artifacts. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1452) to remove hadoop20.jar from lib and use hadoop from the apache maven repo.
[ https://issues.apache.org/jira/browse/PIG-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894807#action_12894807 ] Giridharan Kesavan commented on PIG-1452: - bq. Besides that, I see some more jars pulled along with hadoop, eg: core-3.1.1.jar, ftpserver-deprecated-1.0.0-M2.jar, etc, which we never see before and it drastically increase the size of pig.jar. Do you know what's happen? How can we exclude these jars? These are transient dependencies of hadoop-test.jar to remove hadoop20.jar from lib and use hadoop from the apache maven repo. -- Key: PIG-1452 URL: https://issues.apache.org/jira/browse/PIG-1452 Project: Pig Issue Type: Improvement Components: build Affects Versions: 0.8.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Fix For: 0.8.0 Attachments: PIG-1452.PATCH, PIG-1452V2.PATCH pig use ivy for dependency management. But still it uses hadoop20.jar from the lib folder. Now that we have the hadoop-0.20.2 artifacts available in the maven repo, pig should leverage ivy for resolving/retrieving hadoop artifacts. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1452) to remove hadoop20.jar from lib and use hadoop from the apache maven repo.
[ https://issues.apache.org/jira/browse/PIG-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12894809#action_12894809 ] Giridharan Kesavan commented on PIG-1452: - bq. We only need to exclude three jars which are pulled by Pig: hsqldb.jar jython.jar javacc.jar I remember that hadoop depends on hsqldb.jar. Is there any reason why we dont want it to be added to the pig.jar? to remove hadoop20.jar from lib and use hadoop from the apache maven repo. -- Key: PIG-1452 URL: https://issues.apache.org/jira/browse/PIG-1452 Project: Pig Issue Type: Improvement Components: build Affects Versions: 0.8.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Fix For: 0.8.0 Attachments: PIG-1452.PATCH, PIG-1452V2.PATCH pig use ivy for dependency management. But still it uses hadoop20.jar from the lib folder. Now that we have the hadoop-0.20.2 artifacts available in the maven repo, pig should leverage ivy for resolving/retrieving hadoop artifacts. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1452) to remove hadoop20.jar from lib and use hadoop from the apache maven repo.
[ https://issues.apache.org/jira/browse/PIG-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1452: Status: Patch Available (was: Reopened) to remove hadoop20.jar from lib and use hadoop from the apache maven repo. -- Key: PIG-1452 URL: https://issues.apache.org/jira/browse/PIG-1452 Project: Pig Issue Type: Improvement Components: build Affects Versions: 0.8.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Fix For: 0.8.0 Attachments: PIG-1452.PATCH, PIG-1452V2.PATCH pig use ivy for dependency management. But still it uses hadoop20.jar from the lib folder. Now that we have the hadoop-0.20.2 artifacts available in the maven repo, pig should leverage ivy for resolving/retrieving hadoop artifacts. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1452) to remove hadoop20.jar from lib and use hadoop from the apache maven repo.
[ https://issues.apache.org/jira/browse/PIG-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1452: Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed I just committed this! to remove hadoop20.jar from lib and use hadoop from the apache maven repo. -- Key: PIG-1452 URL: https://issues.apache.org/jira/browse/PIG-1452 Project: Pig Issue Type: Improvement Components: build Affects Versions: 0.8.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Fix For: 0.8.0 Attachments: PIG-1452.PATCH pig use ivy for dependency management. But still it uses hadoop20.jar from the lib folder. Now that we have the hadoop-0.20.2 artifacts available in the maven repo, pig should leverage ivy for resolving/retrieving hadoop artifacts. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (PIG-1500) guava.jar should be removed from the lib folder
guava.jar should be removed from the lib folder --- Key: PIG-1500 URL: https://issues.apache.org/jira/browse/PIG-1500 Project: Pig Issue Type: Bug Components: build Reporter: Giridharan Kesavan guava jar is available in the maven repository but still its is checked into the pig trunk's lib folder. I ve checked the availability of guava jar in the maven repository. http://mvnrepository.com/artifact/com.google.guava/guava -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (PIG-1452) to remove hadoop20.jar from lib and use hadoop from the apache maven repo.
to remove hadoop20.jar from lib and use hadoop from the apache maven repo. -- Key: PIG-1452 URL: https://issues.apache.org/jira/browse/PIG-1452 Project: Pig Issue Type: Improvement Components: build Affects Versions: 0.8.0 Reporter: Giridharan Kesavan pig use ivy for dependency management. But still it uses hadoop20.jar from the lib folder. Now that we have the hadoop-0.20.2 artifacts available in the maven repo, pig should leverage ivy for resolving/retrieving hadoop artifacts. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1452) to remove hadoop20.jar from lib and use hadoop from the apache maven repo.
[ https://issues.apache.org/jira/browse/PIG-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan reassigned PIG-1452: --- Assignee: Giridharan Kesavan to remove hadoop20.jar from lib and use hadoop from the apache maven repo. -- Key: PIG-1452 URL: https://issues.apache.org/jira/browse/PIG-1452 Project: Pig Issue Type: Improvement Components: build Affects Versions: 0.8.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: PIG-1452.PATCH pig use ivy for dependency management. But still it uses hadoop20.jar from the lib folder. Now that we have the hadoop-0.20.2 artifacts available in the maven repo, pig should leverage ivy for resolving/retrieving hadoop artifacts. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1452) to remove hadoop20.jar from lib and use hadoop from the apache maven repo.
[ https://issues.apache.org/jira/browse/PIG-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1452: Attachment: PIG-1452.PATCH this patch deletes the hadoop20.jar from the lib folder and resolves/retrieves hadoop from maven repository using ivy. to remove hadoop20.jar from lib and use hadoop from the apache maven repo. -- Key: PIG-1452 URL: https://issues.apache.org/jira/browse/PIG-1452 Project: Pig Issue Type: Improvement Components: build Affects Versions: 0.8.0 Reporter: Giridharan Kesavan Attachments: PIG-1452.PATCH pig use ivy for dependency management. But still it uses hadoop20.jar from the lib folder. Now that we have the hadoop-0.20.2 artifacts available in the maven repo, pig should leverage ivy for resolving/retrieving hadoop artifacts. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1452) to remove hadoop20.jar from lib and use hadoop from the apache maven repo.
[ https://issues.apache.org/jira/browse/PIG-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1452: Status: Patch Available (was: Open) to remove hadoop20.jar from lib and use hadoop from the apache maven repo. -- Key: PIG-1452 URL: https://issues.apache.org/jira/browse/PIG-1452 Project: Pig Issue Type: Improvement Components: build Affects Versions: 0.8.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: PIG-1452.PATCH pig use ivy for dependency management. But still it uses hadoop20.jar from the lib folder. Now that we have the hadoop-0.20.2 artifacts available in the maven repo, pig should leverage ivy for resolving/retrieving hadoop artifacts. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1452) to remove hadoop20.jar from lib and use hadoop from the apache maven repo.
[ https://issues.apache.org/jira/browse/PIG-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12879478#action_12879478 ] Giridharan Kesavan commented on PIG-1452: - Yes actually hadoop would resolve/retrieve its transient dependencies. But I added these dependencies b'coz pig directly depends on these. When we remove these new set of dependencies pig build would fail even before failing for not having hadoop. (Here ivy is smart enough so it woudnt retrieve these artifacts, though they are transient dependencies of hadoop again) to remove hadoop20.jar from lib and use hadoop from the apache maven repo. -- Key: PIG-1452 URL: https://issues.apache.org/jira/browse/PIG-1452 Project: Pig Issue Type: Improvement Components: build Affects Versions: 0.8.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: PIG-1452.PATCH pig use ivy for dependency management. But still it uses hadoop20.jar from the lib folder. Now that we have the hadoop-0.20.2 artifacts available in the maven repo, pig should leverage ivy for resolving/retrieving hadoop artifacts. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1302) Include zebra's pigtest ant target as a part of pig's ant test target
[ https://issues.apache.org/jira/browse/PIG-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12878904#action_12878904 ] Giridharan Kesavan commented on PIG-1302: - This patch just adds the pigtest target as part of the test target, and doesnt modify any source code, the test failures are not-related. Include zebra's pigtest ant target as a part of pig's ant test target --- Key: PIG-1302 URL: https://issues.apache.org/jira/browse/PIG-1302 Project: Pig Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Pradeep Kamath Assignee: Giridharan Kesavan Attachments: PIG-1302.patch There are changes made in Pig interfaces which break zebra loaders/storers. It would be good to run the pig tests in the zebra unit tests as part of running pig's core-test for each patch submission. So essentially in the test ant target in pig, we would need to invoke zebra's pigtest target. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1302) Include zebra's pigtest ant target as a part of pig's ant test target
[ https://issues.apache.org/jira/browse/PIG-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1302: Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Fix Version/s: 0.8.0 Resolution: Fixed I just committed this. Include zebra's pigtest ant target as a part of pig's ant test target --- Key: PIG-1302 URL: https://issues.apache.org/jira/browse/PIG-1302 Project: Pig Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Pradeep Kamath Assignee: Giridharan Kesavan Fix For: 0.8.0 Attachments: PIG-1302.patch There are changes made in Pig interfaces which break zebra loaders/storers. It would be good to run the pig tests in the zebra unit tests as part of running pig's core-test for each patch submission. So essentially in the test ant target in pig, we would need to invoke zebra's pigtest target. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1302) Include zebra's pigtest ant target as a part of pig's ant test target
[ https://issues.apache.org/jira/browse/PIG-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1302: Status: Patch Available (was: Open) Include zebra's pigtest ant target as a part of pig's ant test target --- Key: PIG-1302 URL: https://issues.apache.org/jira/browse/PIG-1302 Project: Pig Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Pradeep Kamath Assignee: Giridharan Kesavan Attachments: PIG-1302.patch There are changes made in Pig interfaces which break zebra loaders/storers. It would be good to run the pig tests in the zebra unit tests as part of running pig's core-test for each patch submission. So essentially in the test ant target in pig, we would need to invoke zebra's pigtest target. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1302) Include zebra's pigtest ant target as a part of pig's ant test target
[ https://issues.apache.org/jira/browse/PIG-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1302: Attachment: PIG-1302.patch This patch would add the pigtest target as part of ant test. Include zebra's pigtest ant target as a part of pig's ant test target --- Key: PIG-1302 URL: https://issues.apache.org/jira/browse/PIG-1302 Project: Pig Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Pradeep Kamath Attachments: PIG-1302.patch There are changes made in Pig interfaces which break zebra loaders/storers. It would be good to run the pig tests in the zebra unit tests as part of running pig's core-test for each patch submission. So essentially in the test ant target in pig, we would need to invoke zebra's pigtest target. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1302) Include zebra's pigtest ant target as a part of pig's ant test target
[ https://issues.apache.org/jira/browse/PIG-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan reassigned PIG-1302: --- Assignee: Giridharan Kesavan Include zebra's pigtest ant target as a part of pig's ant test target --- Key: PIG-1302 URL: https://issues.apache.org/jira/browse/PIG-1302 Project: Pig Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Pradeep Kamath Assignee: Giridharan Kesavan Attachments: PIG-1302.patch There are changes made in Pig interfaces which break zebra loaders/storers. It would be good to run the pig tests in the zebra unit tests as part of running pig's core-test for each patch submission. So essentially in the test ant target in pig, we would need to invoke zebra's pigtest target. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (PIG-1413) to have a seperate test-patch script for pig patch tests
to have a seperate test-patch script for pig patch tests Key: PIG-1413 URL: https://issues.apache.org/jira/browse/PIG-1413 Project: Pig Issue Type: Sub-task Reporter: Giridharan Kesavan test-patch.sh is shared across hadoop projects through svn:externals, which needs to be removed so that we can have project specific test-patch process. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-1413) to have a seperate test-patch script for pig patch tests
[ https://issues.apache.org/jira/browse/PIG-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan resolved PIG-1413. - Fix Version/s: 0.8.0 Resolution: Fixed Removed the svn:externals reference to test-patch.sh script and created a local copy of test-patch.sh script. to have a seperate test-patch script for pig patch tests Key: PIG-1413 URL: https://issues.apache.org/jira/browse/PIG-1413 Project: Pig Issue Type: Sub-task Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Fix For: 0.8.0 test-patch.sh is shared across hadoop projects through svn:externals, which needs to be removed so that we can have project specific test-patch process. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1413) to have a seperate test-patch script for pig patch tests
[ https://issues.apache.org/jira/browse/PIG-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan reassigned PIG-1413: --- Assignee: Giridharan Kesavan to have a seperate test-patch script for pig patch tests Key: PIG-1413 URL: https://issues.apache.org/jira/browse/PIG-1413 Project: Pig Issue Type: Sub-task Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan test-patch.sh is shared across hadoop projects through svn:externals, which needs to be removed so that we can have project specific test-patch process. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1372) Restore PigInputFormat.sJob for backward compatibility
[ https://issues.apache.org/jira/browse/PIG-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1372: Status: Open (was: Patch Available) Restore PigInputFormat.sJob for backward compatibility -- Key: PIG-1372 URL: https://issues.apache.org/jira/browse/PIG-1372 Project: Pig Issue Type: Bug Affects Versions: 0.7.0 Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: 0.7.0 Attachments: PIG-1372.patch The preferred method to get the job's Configuration object would be to use UDFContext.getJobConf(). This jira is to restore PigInputFormat.sJob (but we will be marking it deprecated and indicating to use UDFContext.getJobConf() instead) to be backward compatible - we can remove it from pig in a future release. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (PIG-1372) Restore PigInputFormat.sJob for backward compatibility
[ https://issues.apache.org/jira/browse/PIG-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1372: Status: Patch Available (was: Open) Restore PigInputFormat.sJob for backward compatibility -- Key: PIG-1372 URL: https://issues.apache.org/jira/browse/PIG-1372 Project: Pig Issue Type: Bug Affects Versions: 0.7.0 Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: 0.7.0 Attachments: PIG-1372.patch The preferred method to get the job's Configuration object would be to use UDFContext.getJobConf(). This jira is to restore PigInputFormat.sJob (but we will be marking it deprecated and indicating to use UDFContext.getJobConf() instead) to be backward compatible - we can remove it from pig in a future release. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (PIG-1372) Restore PigInputFormat.sJob for backward compatibility
[ https://issues.apache.org/jira/browse/PIG-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1372: Status: Open (was: Patch Available) Restore PigInputFormat.sJob for backward compatibility -- Key: PIG-1372 URL: https://issues.apache.org/jira/browse/PIG-1372 Project: Pig Issue Type: Bug Affects Versions: 0.7.0 Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: 0.7.0 Attachments: PIG-1372.patch The preferred method to get the job's Configuration object would be to use UDFContext.getJobConf(). This jira is to restore PigInputFormat.sJob (but we will be marking it deprecated and indicating to use UDFContext.getJobConf() instead) to be backward compatible - we can remove it from pig in a future release. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (PIG-1372) Restore PigInputFormat.sJob for backward compatibility
[ https://issues.apache.org/jira/browse/PIG-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-1372: Status: Patch Available (was: Open) Restore PigInputFormat.sJob for backward compatibility -- Key: PIG-1372 URL: https://issues.apache.org/jira/browse/PIG-1372 Project: Pig Issue Type: Bug Affects Versions: 0.7.0 Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: 0.7.0 Attachments: PIG-1372.patch The preferred method to get the job's Configuration object would be to use UDFContext.getJobConf(). This jira is to restore PigInputFormat.sJob (but we will be marking it deprecated and indicating to use UDFContext.getJobConf() instead) to be backward compatible - we can remove it from pig in a future release. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Sendmail restartd for patch process
Folks, It looks like sendmail process on hudson.zones crashed and all the test patch jobs submitted in the last two days didn't get through. Please resubmit your patch to the test-patch job. -Giri
Re: Runaway Pig builds on hudson.
I ve re-launched the hudson slaves on h7 and h8 , nfsmount points on both the machines seem to have failed. I ve filed a request with the yahoo operations to fix this. Thanks., Giri On 31/03/10 9:11 PM, Patrick Hunt ph...@apache.org wrote: Pig has been running on hudson for 9 and 11 hrs respectively: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/261/ http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/273/ I can't kill it from the console, could someone with greater access than I resolve this? My ZooKeeper builds are hung up waiting for Pig to finish. Thanks! Patrick
[jira] Created: (PIG-1328) pigtest ant target fails pigtrunk builds
pigtest ant target fails pigtrunk builds Key: PIG-1328 URL: https://issues.apache.org/jira/browse/PIG-1328 Project: Pig Issue Type: Bug Components: build Reporter: Giridharan Kesavan java.lang.NoClassDefFoundError:com_cenqua_clover/CloverVersionInfo) [junit] Tests run: 0, Failures: 0, Errors: 2, Time elapsed: 0.154 sec [junit] Test org.apache.hadoop.zebra.pig.TestTableSortStorer FAILED [junit] Running org.apache.hadoop.zebra.pig.TestTableSortStorerDesc [junit] log4j:WARN No appenders could be found for logger (org.apache.hadoop.conf.Configuration). [junit] log4j:WARN Please initialize the log4j system properly. [junit] [CLOVER] FATAL ERROR: Clover could not be initialised. Are you sure you have Clover in the runtime classpath? (class java.lang.NoClassDefFoundError:com_cenqua_clover/CloverVersionInfo) [junit] Tests run: 0, Failures: 0, Errors: 2, Time elapsed: 0.164 sec [junit] Test org.apache.hadoop.zebra.pig.TestTableSortStorerDesc FAILED -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Pig-trunk build 713
Pig-trunk build stuck on h7 machine for more than a day, I ve killed the stuck job and restarted the build on hudson. http://hudson.zones.apache.org/hudson/view/Pig/job/Pig-trunk/713/ -Giri
[jira] Commented: (PIG-996) [zebra] Zebra build script does not have findbugs and clover targets.
[ https://issues.apache.org/jira/browse/PIG-996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12768567#action_12768567 ] Giridharan Kesavan commented on PIG-996: How do we want this findbugs and checkstsyle to be run? with this patch if someone wants to execute findbugs they cant run it from the top level pig project.. Can we not use subant task to run finbugs on zebra as well when findbugs at the root level(pig tunk) is called? [zebra] Zebra build script does not have findbugs and clover targets. - Key: PIG-996 URL: https://issues.apache.org/jira/browse/PIG-996 Project: Pig Issue Type: Bug Components: build Affects Versions: 0.4.0 Reporter: Chao Wang Assignee: Chao Wang Fix For: 0.6.0 Attachments: patch_build, patch_build Zebra build script does not have findbugs and clover targets, leading hudson build process to fail on Zebra. This jira is to fix this by adding these two targets. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-721) redirecting releaseaudit o/p to build/test/releaseaudit/pig-releaseaudit-report.txt
[ https://issues.apache.org/jira/browse/PIG-721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12760544#action_12760544 ] Giridharan Kesavan commented on PIG-721: I understand redirecting releaseaudit warnings to at log file would break the current test patch process tnx! redirecting releaseaudit o/p to build/test/releaseaudit/pig-releaseaudit-report.txt Key: PIG-721 URL: https://issues.apache.org/jira/browse/PIG-721 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: PIG-721.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-942) Maps are not implicitly casted
[ https://issues.apache.org/jira/browse/PIG-942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-942: --- Status: Patch Available (was: Open) resubmiting patch to hudson Maps are not implicitly casted -- Key: PIG-942 URL: https://issues.apache.org/jira/browse/PIG-942 Project: Pig Issue Type: Bug Reporter: Sriranjan Manjunath Assignee: Pradeep Kamath Fix For: 0.6.0 Attachments: PIG-942-2.patch, PIG-942.patch A = load 'foo' as (m) throws the following exception when foo has maps. java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.util.Map at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:98) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:115) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:612) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:278) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:240) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:249) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:240) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:93) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198) The same works if I explicitly cast m to a map: A = load 'foo' as (m:[]) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-942) Maps are not implicitly casted
[ https://issues.apache.org/jira/browse/PIG-942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-942: --- Status: Open (was: Patch Available) Maps are not implicitly casted -- Key: PIG-942 URL: https://issues.apache.org/jira/browse/PIG-942 Project: Pig Issue Type: Bug Reporter: Sriranjan Manjunath Assignee: Pradeep Kamath Fix For: 0.6.0 Attachments: PIG-942-2.patch, PIG-942.patch A = load 'foo' as (m) throws the following exception when foo has maps. java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.util.Map at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:98) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:115) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:612) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:278) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:240) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:249) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:240) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:93) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198) The same works if I explicitly cast m to a map: A = load 'foo' as (m:[]) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-942) Maps are not implicitly casted
[ https://issues.apache.org/jira/browse/PIG-942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-942: --- Status: Open (was: Patch Available) Maps are not implicitly casted -- Key: PIG-942 URL: https://issues.apache.org/jira/browse/PIG-942 Project: Pig Issue Type: Bug Reporter: Sriranjan Manjunath Assignee: Pradeep Kamath Fix For: 0.6.0 Attachments: PIG-942-2.patch, PIG-942.patch A = load 'foo' as (m) throws the following exception when foo has maps. java.lang.ClassCastException: org.apache.pig.data.DataByteArray cannot be cast to java.util.Map at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:98) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:115) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:612) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:278) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:240) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:249) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:240) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:93) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198) The same works if I explicitly cast m to a map: A = load 'foo' as (m:[]) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-939) Checkstyle pulls in junit3.7 which causes the build of test code to fail.
[
https://issues.apache.org/jira/browse/PIG-939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giridharan Kesavan resolved PIG-939.
Resolution: Fixed
Fix Version/s: 0.4.0
Checkstyle pulls in junit3.7 which causes the build of test code to fail.
-
Key: PIG-939
URL: https://issues.apache.org/jira/browse/PIG-939
Project: Pig
Issue Type: Bug
Components: build
Affects Versions: 0.3.0
Reporter: Lee Tucker
Assignee: Giridharan Kesavan
Fix For: 0.4.0
Attachments: pig-939.patch
Pig fails to compile if you execute:
ant -Dassociated flags for various components clean findbugs checkstyle
test
It gets the error:
[javac] Compiling 153 source files to
/export/crawlspace/kryptonite/hadoopqa/workspace/workspace/CCDI-Pig-2.3/pig-2.3.0.0.20.0.2967040009/build/test/classes
[javac]
/export/crawlspace/kryptonite/hadoopqa/workspace/workspace/CCDI-Pig-2.3/pig-2.3.0.0.20.0.2967040009/test/org/apache/pig/test/PigExecTestCase.java:31:
cannot find symbol
[javac] symbol : constructor TestCase()
[javac] location: class junit.framework.TestCase
[javac] public abstract class PigExecTestCase extends TestCase {
[javac] ^
Once that's done, there's a copy of junit 3.7 cached from ivy that will
continue to cause the build to fail. It will succeed, if you remove it, and
then do:
ant -Dassociated flags for various components clean findbugs test
This proves it's running checkstyle that pulls in junit 3.7
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Status: Patch Available (was: Open) to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Attachment: pig-765.patch updated patch for pig-0.4.0-dev to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12752457#action_12752457 ] Giridharan Kesavan commented on PIG-765: - 1 for releaseaudit warnings is expected as the jdiff created html docs on the fly which wont be having required AL header. - 1 for core test in not related to this patch. to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Status: Open (was: Patch Available) to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Status: Patch Available (was: Open) to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Status: Open (was: Patch Available) to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Status: Patch Available (was: Open) to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-939) Checkstyle pulls in junit3.7 which causes the build of test code to fail.
[
https://issues.apache.org/jira/browse/PIG-939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12752658#action_12752658
]
Giridharan Kesavan commented on PIG-939:
Its not pig which depends on antlr but checkstyle depends on antlr and thats
the reason antlr is defined under conf=checkstyle-master
http://mvnrepository.com/artifact/checkstyle/checkstyle/4.2 - has the details
of checkstyle's transient dependency.
Checkstyle pulls in junit3.7 which causes the build of test code to fail.
-
Key: PIG-939
URL: https://issues.apache.org/jira/browse/PIG-939
Project: Pig
Issue Type: Bug
Components: build
Affects Versions: 0.3.0
Reporter: Lee Tucker
Assignee: Giridharan Kesavan
Attachments: pig-939.patch
Pig fails to compile if you execute:
ant -Dassociated flags for various components clean findbugs checkstyle
test
It gets the error:
[javac] Compiling 153 source files to
/export/crawlspace/kryptonite/hadoopqa/workspace/workspace/CCDI-Pig-2.3/pig-2.3.0.0.20.0.2967040009/build/test/classes
[javac]
/export/crawlspace/kryptonite/hadoopqa/workspace/workspace/CCDI-Pig-2.3/pig-2.3.0.0.20.0.2967040009/test/org/apache/pig/test/PigExecTestCase.java:31:
cannot find symbol
[javac] symbol : constructor TestCase()
[javac] location: class junit.framework.TestCase
[javac] public abstract class PigExecTestCase extends TestCase {
[javac] ^
Once that's done, there's a copy of junit 3.7 cached from ivy that will
continue to cause the build to fail. It will succeed, if you remove it, and
then do:
ant -Dassociated flags for various components clean findbugs test
This proves it's running checkstyle that pulls in junit 3.7
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12752906#action_12752906 ] Giridharan Kesavan commented on PIG-765: tests passed on my local box; anyways I 've changed to the way classpath references are made in the build.xml to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-939) Checkstyle pulls in junit3.7 which causes the build of test code to fail.
[
https://issues.apache.org/jira/browse/PIG-939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giridharan Kesavan reassigned PIG-939:
--
Assignee: Giridharan Kesavan
Checkstyle pulls in junit3.7 which causes the build of test code to fail.
-
Key: PIG-939
URL: https://issues.apache.org/jira/browse/PIG-939
Project: Pig
Issue Type: Bug
Components: build
Affects Versions: 0.3.0
Reporter: Lee Tucker
Assignee: Giridharan Kesavan
Attachments: pig-939.patch
Pig fails to compile if you execute:
ant -Dassociated flags for various components clean findbugs checkstyle
test
It gets the error:
[javac] Compiling 153 source files to
/export/crawlspace/kryptonite/hadoopqa/workspace/workspace/CCDI-Pig-2.3/pig-2.3.0.0.20.0.2967040009/build/test/classes
[javac]
/export/crawlspace/kryptonite/hadoopqa/workspace/workspace/CCDI-Pig-2.3/pig-2.3.0.0.20.0.2967040009/test/org/apache/pig/test/PigExecTestCase.java:31:
cannot find symbol
[javac] symbol : constructor TestCase()
[javac] location: class junit.framework.TestCase
[javac] public abstract class PigExecTestCase extends TestCase {
[javac] ^
Once that's done, there's a copy of junit 3.7 cached from ivy that will
continue to cause the build to fail. It will succeed, if you remove it, and
then do:
ant -Dassociated flags for various components clean findbugs test
This proves it's running checkstyle that pulls in junit 3.7
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-939) Checkstyle pulls in junit3.7 which causes the build of test code to fail.
[
https://issues.apache.org/jira/browse/PIG-939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giridharan Kesavan updated PIG-939:
---
Status: Patch Available (was: Open)
Checkstyle pulls in junit3.7 which causes the build of test code to fail.
-
Key: PIG-939
URL: https://issues.apache.org/jira/browse/PIG-939
Project: Pig
Issue Type: Bug
Components: build
Affects Versions: 0.3.0
Reporter: Lee Tucker
Attachments: pig-939.patch
Pig fails to compile if you execute:
ant -Dassociated flags for various components clean findbugs checkstyle
test
It gets the error:
[javac] Compiling 153 source files to
/export/crawlspace/kryptonite/hadoopqa/workspace/workspace/CCDI-Pig-2.3/pig-2.3.0.0.20.0.2967040009/build/test/classes
[javac]
/export/crawlspace/kryptonite/hadoopqa/workspace/workspace/CCDI-Pig-2.3/pig-2.3.0.0.20.0.2967040009/test/org/apache/pig/test/PigExecTestCase.java:31:
cannot find symbol
[javac] symbol : constructor TestCase()
[javac] location: class junit.framework.TestCase
[javac] public abstract class PigExecTestCase extends TestCase {
[javac] ^
Once that's done, there's a copy of junit 3.7 cached from ivy that will
continue to cause the build to fail. It will succeed, if you remove it, and
then do:
ant -Dassociated flags for various components clean findbugs test
This proves it's running checkstyle that pulls in junit 3.7
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-939) Checkstyle pulls in junit3.7 which causes the build of test code to fail.
[
https://issues.apache.org/jira/browse/PIG-939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giridharan Kesavan updated PIG-939:
---
Attachment: pig-939.patch
this patch should fix this issue of downloading junit-3.7
Checkstyle pulls in junit3.7 which causes the build of test code to fail.
-
Key: PIG-939
URL: https://issues.apache.org/jira/browse/PIG-939
Project: Pig
Issue Type: Bug
Components: build
Affects Versions: 0.3.0
Reporter: Lee Tucker
Attachments: pig-939.patch
Pig fails to compile if you execute:
ant -Dassociated flags for various components clean findbugs checkstyle
test
It gets the error:
[javac] Compiling 153 source files to
/export/crawlspace/kryptonite/hadoopqa/workspace/workspace/CCDI-Pig-2.3/pig-2.3.0.0.20.0.2967040009/build/test/classes
[javac]
/export/crawlspace/kryptonite/hadoopqa/workspace/workspace/CCDI-Pig-2.3/pig-2.3.0.0.20.0.2967040009/test/org/apache/pig/test/PigExecTestCase.java:31:
cannot find symbol
[javac] symbol : constructor TestCase()
[javac] location: class junit.framework.TestCase
[javac] public abstract class PigExecTestCase extends TestCase {
[javac] ^
Once that's done, there's a copy of junit 3.7 cached from ivy that will
continue to cause the build to fail. It will succeed, if you remove it, and
then do:
ant -Dassociated flags for various components clean findbugs test
This proves it's running checkstyle that pulls in junit 3.7
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
pig tunk / patch builds
Pig patch builds are now moved from minerva.apache.org to h7.grid.sp2.yahoo.net machine. h7.grid.sp2.yahoo.net machine is dedicated to run pig builds. -Giri -Original Message- From: Giridharan Kesavan [mailto:gkesa...@yahoo-inc.com] Sent: Wednesday, August 26, 2009 11:05 PM To: pig-dev@hadoop.apache.org Subject: RE: pig trunk build Its only the trunk build that I moved to h7 AND NOT THE PATCH BUILDS. And this is not going to change anything from the dev point of view. Trunk build url still remains the same: http://hudson.zones.apache.org/hudson/view/Pig/job/Pig-trunk/ Patch builds still run on minerva and it has the same url as before. Will keep you posted as I move the patch builds. -Original Message- From: Pradeep Kamath [mailto:prade...@yahoo-inc.com] Sent: Wednesday, August 26, 2009 10:37 PM To: pig-dev@hadoop.apache.org Subject: RE: pig trunk build What is the URL for the Hudson UI? I tried http://hudson.zones.apache.org/hudson/job/Pig-Patch- h7.grid.sp2.yahoo.ne t but that did not work. Pradeep -Original Message- From: Giridharan Kesavan [mailto:gkesa...@yahoo-inc.com] Sent: Wednesday, August 26, 2009 7:41 AM To: pig-dev@hadoop.apache.org Subject: pig trunk build Pig trunk build is now moved from minerva.apache.org to h7.grid.sp2.yahoo.net machine. tnx, Giri
pig trunk build
Pig trunk build is now moved from minerva.apache.org to h7.grid.sp2.yahoo.net machine. tnx, Giri
RE: pig trunk build
Its only the trunk build that I moved to h7 AND NOT THE PATCH BUILDS. And this is not going to change anything from the dev point of view. Trunk build url still remains the same: http://hudson.zones.apache.org/hudson/view/Pig/job/Pig-trunk/ Patch builds still run on minerva and it has the same url as before. Will keep you posted as I move the patch builds. -Original Message- From: Pradeep Kamath [mailto:prade...@yahoo-inc.com] Sent: Wednesday, August 26, 2009 10:37 PM To: pig-dev@hadoop.apache.org Subject: RE: pig trunk build What is the URL for the Hudson UI? I tried http://hudson.zones.apache.org/hudson/job/Pig-Patch- h7.grid.sp2.yahoo.ne t but that did not work. Pradeep -Original Message- From: Giridharan Kesavan [mailto:gkesa...@yahoo-inc.com] Sent: Wednesday, August 26, 2009 7:41 AM To: pig-dev@hadoop.apache.org Subject: pig trunk build Pig trunk build is now moved from minerva.apache.org to h7.grid.sp2.yahoo.net machine. tnx, Giri
[jira] Updated: (PIG-901) InputSplit (SliceWrapper) created by Pig is big in size due to serialized PigContext
[ https://issues.apache.org/jira/browse/PIG-901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-901: --- Status: Open (was: Patch Available) resubmitting to hudson patch queue InputSplit (SliceWrapper) created by Pig is big in size due to serialized PigContext Key: PIG-901 URL: https://issues.apache.org/jira/browse/PIG-901 Project: Pig Issue Type: Bug Affects Versions: 0.3.1 Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: 0.4.0 Attachments: PIG-901-1.patch, PIG-901-branch-0.3.patch, PIG-901-trunk.patch InputSplit (SliceWrapper) created by Pig is big in size due to serialized PigContext. SliceWrapper only needs ExecType - so the entire PigContext should not be serialized and only the ExecType should be serialized. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-901) InputSplit (SliceWrapper) created by Pig is big in size due to serialized PigContext
[ https://issues.apache.org/jira/browse/PIG-901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-901: --- Status: Patch Available (was: Open) InputSplit (SliceWrapper) created by Pig is big in size due to serialized PigContext Key: PIG-901 URL: https://issues.apache.org/jira/browse/PIG-901 Project: Pig Issue Type: Bug Affects Versions: 0.3.1 Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: 0.4.0 Attachments: PIG-901-1.patch, PIG-901-branch-0.3.patch, PIG-901-trunk.patch InputSplit (SliceWrapper) created by Pig is big in size due to serialized PigContext. SliceWrapper only needs ExecType - so the entire PigContext should not be serialized and only the ExecType should be serialized. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
pig patch test builds
Sendmail issue on hudson.zones is fixed and the pig patch build is restarted. Thanks, -Giri
RE: pig patch test builds
If you have changed the jira status to patch available in the last couple of days please resubmit your patch for hudson to pick your patch for testing. -Giri From: Giridharan Kesavan Sent: Wednesday, August 05, 2009 7:31 PM To: 'pig-dev@hadoop.apache.org' Cc: Nigel Daley; Olga Natkovich Subject: pig patch test builds Sendmail issue on hudson.zones is fixed and the pig patch build is restarted. Thanks, -Giri
build failures on hudson zones
Build on hudson.zones are failing as the zonestorage for hudson is full. I 've sent an email to the ASF infra team about the space issues on hudson zones. Once the issues is resolved I would restart hudson for builds. Thanks, Giri
zookeeper patch builds
Looks like hudson space issue is resolved; I 've restarted the zookeeper patch build jobs. -Giri
[jira] Updated: (PIG-697) Proposed improvements to pig's optimizer
[
https://issues.apache.org/jira/browse/PIG-697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giridharan Kesavan updated PIG-697:
---
Status: Open (was: Patch Available)
Proposed improvements to pig's optimizer
Key: PIG-697
URL: https://issues.apache.org/jira/browse/PIG-697
Project: Pig
Issue Type: Bug
Components: impl
Reporter: Alan Gates
Assignee: Santhosh Srinivasan
Attachments: OptimizerPhase1.patch, OptimizerPhase1_part2.patch,
OptimizerPhase2.patch, OptimizerPhase3_parrt1-1.patch,
OptimizerPhase3_parrt1.patch, OptimizerPhase3_part2_3.patch,
OptimizerPhase4_part1-1.patch, OptimizerPhase4_part2.patch
I propose the following changes to pig optimizer, plan, and operator
functionality to support more robust optimization:
1) Remove the required array from Rule. This will change rules so that they
only match exact patterns instead of allowing missing elements in the pattern.
This has the downside that if a given rule applies to two patterns (say
Load-Filter-Group, Load-Group) you have to write two rules. But it has
the upside that
the resulting rules know exactly what they are getting. The original intent
of this was to reduce the number of rules that needed to be written. But the
resulting rules have do a lot of work to understand the operators they are
working with. With exact matches only, each rule will know exactly the
operators it
is working on and can apply the logic of shifting the operators around. All
four of the existing rules set all entries of required to true, so removing
this
will have no effect on them.
2) Change PlanOptimizer.optimize to iterate over the rules until there are no
conversions or a certain number of iterations has been reached. Currently the
function is:
{code}
public final void optimize() throws OptimizerException {
RuleMatcher matcher = new RuleMatcher();
for (Rule rule : mRules) {
if (matcher.match(rule)) {
// It matches the pattern. Now check if the transformer
// approves as well.
ListListO matches = matcher.getAllMatches();
for (ListO match:matches)
{
if (rule.transformer.check(match)) {
// The transformer approves.
rule.transformer.transform(match);
}
}
}
}
}
{code}
It would change to be:
{code}
public final void optimize() throws OptimizerException {
RuleMatcher matcher = new RuleMatcher();
boolean sawMatch;
int iterators = 0;
do {
sawMatch = false;
for (Rule rule : mRules) {
ListListO matches = matcher.getAllMatches();
for (ListO match:matches) {
// It matches the pattern. Now check if the transformer
// approves as well.
if (rule.transformer.check(match)) {
// The transformer approves.
sawMatch = true;
rule.transformer.transform(match);
}
}
}
// Not sure if 1000 is the right number of iterations, maybe it
// should be configurable so that large scripts don't stop too
// early.
} while (sawMatch numIterations++ 1000);
}
{code}
The reason for limiting the number of iterations is to avoid infinite loops.
The reason for iterating over the rules is so that each rule can be applied
multiple
times as necessary. This allows us to write simple rules, mostly swaps
between neighboring operators, without worrying that we get the plan right in
one pass.
For example, we might have a plan that looks like:
Load-Join-Filter-Foreach, and we want to optimize it to
Load-Foreach-Filter-Join. With two simple
rules (swap filter and join and swap foreach and filter), applied
iteratively, we can get from the initial to final plan, without needing to
understanding the
big picture of the entire plan.
3) Add three calls to OperatorPlan:
{code}
/**
* Swap two operators in a plan. Both of the operators must have single
* inputs and single outputs.
* @param first operator
* @param second operator
* @throws PlanException if either operator is not single input and output.
*/
public void swap(E first, E second) throws PlanException {
...
}
/**
* Push one operator in front of another. This function is for use when
* the first operator has multiple inputs. The caller can specify
* which input of the first operator the second operator should be pushed to.
* @param first operator
[jira] Updated: (PIG-839) incorrect return codes on failure when using -f or -e flags
[ https://issues.apache.org/jira/browse/PIG-839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-839: --- Status: Open (was: Patch Available) resubmitting the patch incorrect return codes on failure when using -f or -e flags --- Key: PIG-839 URL: https://issues.apache.org/jira/browse/PIG-839 Project: Pig Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: fix_return_code.patch To repro: pig -e a = load 'some file' ; b = stream a through \`false\` ; store b into 'some file'; Both the -e and -f flags do not return the right code upon exit. Running the script w/o using -f works fine. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-835) Multiquery optimization does not handle the case where the map keys in the split plans have different key types (tuple and non tuple key type)
[
https://issues.apache.org/jira/browse/PIG-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giridharan Kesavan updated PIG-835:
---
Status: Patch Available (was: Open)
resubmitting the patch
Multiquery optimization does not handle the case where the map keys in the
split plans have different key types (tuple and non tuple key type)
--
Key: PIG-835
URL: https://issues.apache.org/jira/browse/PIG-835
Project: Pig
Issue Type: Bug
Affects Versions: 0.2.1
Reporter: Pradeep Kamath
Assignee: Pradeep Kamath
Fix For: 0.3.0
Attachments: PIG-835.patch
A query like the following results in an exception on execution:
{noformat}
a = load 'mult.input' as (name, age, gpa);
b = group a ALL;
c = foreach b generate group, COUNT(a);
store c into 'foo';
d = group a by (name, gpa);
e = foreach d generate flatten(group), MIN(a.age);
store e into 'bar';
{noformat}
Exception on execution:
09/06/04 16:56:11 INFO mapred.TaskInProgress: Error from
attempt_200906041655_0001_r_00_3: java.lang.ClassCastException:
java.lang.String cannot be cast to org.apache.pig.data.Tuple
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:312)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getNext(POStore.java:117)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux.runPipeline(PODemux.java:248)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PODemux.getNext(PODemux.java:238)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:320)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:288)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:268)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:142)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Status: In Progress (was: Patch Available) to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Attachment: pig-765.patch this jdiff patch is created after resolving the author tag issue mentioned in pig-806. to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-564) Parameter Substitution using -param option does not seem to work when parameters contain special characters such as +,=,-,?,'
[
https://issues.apache.org/jira/browse/PIG-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12715792#action_12715792
]
Giridharan Kesavan commented on PIG-564:
Use this link for releaseaudit warnings:
http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/68/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
I 've fixed the test-patch scripts for the broken link.
Parameter Substitution using -param option does not seem to work when
parameters contain special characters such as +,=,-,?,'
---
Key: PIG-564
URL: https://issues.apache.org/jira/browse/PIG-564
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.2.0
Reporter: Viraj Bhat
Assignee: Olga Natkovich
Attachments: PIG-564.patch
Consider the following Pig script which uses parameter substitution
{code}
%default qual '/user/viraj'
%default mydir 'mydir_myextraqual'
VISIT_LOGS = load '$qual/$mydir' as (a,b,c);
dump VISIT_LOGS;
{code}
If you run the script as:
==
java -cp pig.jar:${HADOOP_HOME}/conf/ -Dhod.server='' org.apache.pig.Main
-param mydir=mydir-myextraqual mypigparamsub.pig
==
You get the following error:
==
2008-12-15 19:49:43,964 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- java.io.IOException: /user/viraj/mydir does not exist
at
org.apache.pig.backend.executionengine.PigSlicer.validate(PigSlicer.java:109)
at
org.apache.pig.impl.io.ValidatingInputFileSpec.validate(ValidatingInputFileSpec.java:59)
at
org.apache.pig.impl.io.ValidatingInputFileSpec.init(ValidatingInputFileSpec.java:44)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:200)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:742)
at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:370)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
at
org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
at java.lang.Thread.run(Thread.java:619)
java.io.IOException: Unable to open iterator for alias: VISIT_LOGS [Job
terminated with anomalous status FAILED]
at org.apache.pig.PigServer.openIterator(PigServer.java:389)
at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:269)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:178)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
at org.apache.pig.Main.main(Main.java:306)
Caused by: java.io.IOException: Job terminated with anomalous status FAILED
... 6 more
==
Also tried using: -param mydir='mydir\-myextraqual'
This behavior occurs if the parameter value contains characters such as +,=,
?.
A workaround for this behavior is using a param_file which contains
param_name=param_value on each line, with the param_value enclosed by
quotes. For example:
mydir='mydir-myextraqual' and then running the pig script as:
java -cp pig.jar:${HADOOP_HOME}/conf/ -Dhod.server='' org.apache.pig.Main
-param_file myparamfile mypigparamsub.pig
The following issues need to be fixed:
1) In -param option if parameter value contains special characters, it is
truncated
2) In param_file, if param_value contains a special characters, it should be
enclosed in quotes
3) If 2 is a known issue then it should be documented in
http://wiki.apache.org/pig/ParameterSubstitution
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-701) Implement IVY for resolving pig dependencies
[ https://issues.apache.org/jira/browse/PIG-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-701: --- Resolution: Fixed Fix Version/s: 0.3.0 Status: Resolved (was: Patch Available) Implement IVY for resolving pig dependencies - Key: PIG-701 URL: https://issues.apache.org/jira/browse/PIG-701 Project: Pig Issue Type: New Feature Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Fix For: 0.3.0 Attachments: lib_rm.sh, PIG-701.patch, PIG-701.patch, PIG-701.patch, PIG-701.patch pig libraries to be resolved using IVY from the centralized maven repository. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Hadoop Flags: [Reviewed] Status: Patch Available (was: In Progress) to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Status: In Progress (was: Patch Available) to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-806) to remove author tags in the pig source code
[ https://issues.apache.org/jira/browse/PIG-806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12708422#action_12708422 ] Giridharan Kesavan commented on PIG-806: This issue blocks : https://issues.apache.org/jira/browse/PIG-765 to remove author tags in the pig source code Key: PIG-806 URL: https://issues.apache.org/jira/browse/PIG-806 Project: Pig Issue Type: Bug Reporter: Giridharan Kesavan Following java source files has author tags in them ; which need to to be cleaned. src/org/apache/pig/Algebraic.java src/org/apache/pig/backend/local/executionengine/physicalLayer/relationalOperators/POCross.java src/org/apache/pig/backend/local/executionengine/physicalLayer/relationalOperators/POCogroup.java src/org/apache/pig/impl/io/FileSpec.java src/org/apache/pig/impl/streaming/StreamingCommand.java src/org/apache/pig/StoreFunc.java src/org/apache/pig/tools/cmdline/CmdLineParser.java src/org/apache/pig/tools/timer/PerformanceTimer.java src/org/apache/pig/tools/timer/PerformanceTimerFactory.java Thanks, -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Attachment: pig-765.patch ported patch to resolve the jdiff dependencies using ivy. tnx! to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch, pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-794) Use Avro serialization in Pig
[ https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-794: --- Attachment: PIG-794.patch this patch resolves jackson-asl.jar from the mvn repo through ivy and avro from the local lib dir. While submitting this patch to svn we have to add avro jar to the lib dir tnx! Use Avro serialization in Pig - Key: PIG-794 URL: https://issues.apache.org/jira/browse/PIG-794 Project: Pig Issue Type: Improvement Components: impl Affects Versions: 0.2.0 Reporter: Rakesh Setty Fix For: 0.2.0 Attachments: avro-0.1-dev-java.jar, AvroStorage.patch, jackson-asl-0.9.4.jar, PIG-794.patch We would like to use Avro serialization in Pig to pass data between MR jobs instead of the current BinStorage. Attached is an implementation of AvroBinStorage which performs significantly better compared to BinStorage on our benchmarks. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-701) Implement IVY for resolving pig dependencies
[ https://issues.apache.org/jira/browse/PIG-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-701: --- Status: Patch Available (was: Open) Implement IVY for resolving pig dependencies - Key: PIG-701 URL: https://issues.apache.org/jira/browse/PIG-701 Project: Pig Issue Type: New Feature Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: lib_rm.sh, PIG-701.patch, PIG-701.patch, PIG-701.patch, PIG-701.patch pig libraries to be resolved using IVY from the centralized maven repository. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-701) Implement IVY for resolving pig dependencies
[ https://issues.apache.org/jira/browse/PIG-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-701: --- Status: Open (was: Patch Available) resubmitting a patch Implement IVY for resolving pig dependencies - Key: PIG-701 URL: https://issues.apache.org/jira/browse/PIG-701 Project: Pig Issue Type: New Feature Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: lib_rm.sh, PIG-701.patch, PIG-701.patch, PIG-701.patch, PIG-701.patch pig libraries to be resolved using IVY from the centralized maven repository. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-701) Implement IVY for resolving pig dependencies
[ https://issues.apache.org/jira/browse/PIG-701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12705568#action_12705568 ] Giridharan Kesavan commented on PIG-701: I just committed this! Implement IVY for resolving pig dependencies - Key: PIG-701 URL: https://issues.apache.org/jira/browse/PIG-701 Project: Pig Issue Type: New Feature Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: lib_rm.sh, PIG-701.patch, PIG-701.patch, PIG-701.patch, PIG-701.patch pig libraries to be resolved using IVY from the centralized maven repository. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-701) Implement IVY for resolving pig dependencies
[ https://issues.apache.org/jira/browse/PIG-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-701: --- Attachment: PIG-701.patch fixed those javac warnings and findbugs warnings! Implement IVY for resolving pig dependencies - Key: PIG-701 URL: https://issues.apache.org/jira/browse/PIG-701 Project: Pig Issue Type: New Feature Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: lib_rm.sh, PIG-701.patch, PIG-701.patch, PIG-701.patch pig libraries to be resolved using IVY from the centralized maven repository. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Status: Patch Available (was: Open) to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-765) to implement jdiff
[ https://issues.apache.org/jira/browse/PIG-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-765: --- Attachment: pig-765.patch this patch implements jdiff. to implement jdiff -- Key: PIG-765 URL: https://issues.apache.org/jira/browse/PIG-765 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-765.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-762) to move the test logs and hadoop job logs to diff location
[ https://issues.apache.org/jira/browse/PIG-762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-762: --- Status: Open (was: Patch Available) to move the test logs and hadoop job logs to diff location --- Key: PIG-762 URL: https://issues.apache.org/jira/browse/PIG-762 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-762.patch junit test logs and hadoop logs are put in the same build/test/logs folder .. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-762) to move the test logs and hadoop job logs to diff location
[ https://issues.apache.org/jira/browse/PIG-762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-762: --- Attachment: pig-762.patch my bad; recreating the patch thanks Santosh. to move the test logs and hadoop job logs to diff location --- Key: PIG-762 URL: https://issues.apache.org/jira/browse/PIG-762 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-762.patch, pig-762.patch junit test logs and hadoop logs are put in the same build/test/logs folder .. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (PIG-764) pig patch tests are failing
pig patch tests are failing Key: PIG-764 URL: https://issues.apache.org/jira/browse/PIG-764 Project: Pig Issue Type: Bug Reporter: Giridharan Kesavan Pig patch tests are failing... http://hudson.zones.apache.org/hudson/view/Pig/job/Pig-Patch-minerva.apache.org/lastBuild/testReport/ -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-762) to move the test logs and hadoop job logs to diff location
[
https://issues.apache.org/jira/browse/PIG-762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12698660#action_12698660
]
Giridharan Kesavan commented on PIG-762:
{quote}
09/04/09 04:30:48 ERROR mapReduceLayer.Launcher: java.lang.Exception:
org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access
`/tmp/hadoop-hudson/mapred/system/job_local_0002': No such file or directory
Could this be a problem with setup on Hudson?
{quote}
nope, there is no conf change on hudson;
this test patch build is configured in the same way as the pig trunk build.
I 've filed pig-764 for tracking this issue
Thanks,
to move the test logs and hadoop job logs to diff location
---
Key: PIG-762
URL: https://issues.apache.org/jira/browse/PIG-762
Project: Pig
Issue Type: Improvement
Components: build
Reporter: Giridharan Kesavan
Assignee: Giridharan Kesavan
Attachments: pig-762.patch, pig-762.patch
junit test logs and hadoop logs are put in the same build/test/logs folder ..
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-762) to move the test logs and hadoop job logs to diff location
[ https://issues.apache.org/jira/browse/PIG-762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-762: --- Attachment: pig-762.patch this path would put the hadoop logs to build/test/logs/hadoop dir and the test logs to build/test/logs dir .. Santosh, Could you please review this patch ? Thanks, to move the test logs and hadoop job logs to diff location --- Key: PIG-762 URL: https://issues.apache.org/jira/browse/PIG-762 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-762.patch junit test logs and hadoop logs are put in the same build/test/logs folder .. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-762) to move the test logs and hadoop job logs to diff location
[ https://issues.apache.org/jira/browse/PIG-762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-762: --- Status: Patch Available (was: Open) to move the test logs and hadoop job logs to diff location --- Key: PIG-762 URL: https://issues.apache.org/jira/browse/PIG-762 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-762.patch junit test logs and hadoop logs are put in the same build/test/logs folder .. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (PIG-762) to move the test logs and hadoop job logs to diff location
to move the test logs and hadoop job logs to diff location --- Key: PIG-762 URL: https://issues.apache.org/jira/browse/PIG-762 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan junit test logs and hadoop logs are put in the same build/test/logs folder .. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-733) Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input
[ https://issues.apache.org/jira/browse/PIG-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12696416#action_12696416 ] Giridharan Kesavan commented on PIG-733: I'm going to resubmit the patch to hudson .. Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input --- Key: PIG-733 URL: https://issues.apache.org/jira/browse/PIG-733 Project: Pig Issue Type: Bug Affects Versions: 0.2.0 Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: 0.3.0 Attachments: PIG-733-v2.patch, PIG-733.patch Order by has a sampling job which samples the input and creates a sorted list of sample items. CUrrently the number of items sampled is 100 per map task. So if the input is large resulting in many maps (say 50,000) the sample is big. This sorted sample is stored on dfs. The WeightedRangePartitioner computes quantile boundaries and weighted probabilities for repeating values in each map by reading the samples file from DFS. In queries with many maps (in the order of 50,000) the dfs read of the sample file fails with FileSystem closed error. This seems to point to a dfs issue wherein a big dfs file being read simultaneously by many dfs clients (in this case all maps) causes the clients to be closed. However on the pig side, loading the sample from each map in the final map reduce job and computing the quantile boundaries and weighted probabilities is inefficient. We should do this computation through a FindQuantiles udf in the same map reduce job which produces the sorted samples. This way lesser data is written to dfs and in the final map reduce job, the weightedRangePartitioner needs to just load the computed information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-733) Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input
[ https://issues.apache.org/jira/browse/PIG-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-733: --- Status: Patch Available (was: Open) Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input --- Key: PIG-733 URL: https://issues.apache.org/jira/browse/PIG-733 Project: Pig Issue Type: Bug Affects Versions: 0.2.0 Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: 0.3.0 Attachments: PIG-733-v2.patch, PIG-733.patch Order by has a sampling job which samples the input and creates a sorted list of sample items. CUrrently the number of items sampled is 100 per map task. So if the input is large resulting in many maps (say 50,000) the sample is big. This sorted sample is stored on dfs. The WeightedRangePartitioner computes quantile boundaries and weighted probabilities for repeating values in each map by reading the samples file from DFS. In queries with many maps (in the order of 50,000) the dfs read of the sample file fails with FileSystem closed error. This seems to point to a dfs issue wherein a big dfs file being read simultaneously by many dfs clients (in this case all maps) causes the clients to be closed. However on the pig side, loading the sample from each map in the final map reduce job and computing the quantile boundaries and weighted probabilities is inefficient. We should do this computation through a FindQuantiles udf in the same map reduce job which produces the sorted samples. This way lesser data is written to dfs and in the final map reduce job, the weightedRangePartitioner needs to just load the computed information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-733) Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input
[ https://issues.apache.org/jira/browse/PIG-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-733: --- Status: Open (was: Patch Available) Order by sampling dumps entire sample to hdfs which causes dfs FileSystem closed error on large input --- Key: PIG-733 URL: https://issues.apache.org/jira/browse/PIG-733 Project: Pig Issue Type: Bug Affects Versions: 0.2.0 Reporter: Pradeep Kamath Assignee: Pradeep Kamath Fix For: 0.3.0 Attachments: PIG-733-v2.patch, PIG-733.patch Order by has a sampling job which samples the input and creates a sorted list of sample items. CUrrently the number of items sampled is 100 per map task. So if the input is large resulting in many maps (say 50,000) the sample is big. This sorted sample is stored on dfs. The WeightedRangePartitioner computes quantile boundaries and weighted probabilities for repeating values in each map by reading the samples file from DFS. In queries with many maps (in the order of 50,000) the dfs read of the sample file fails with FileSystem closed error. This seems to point to a dfs issue wherein a big dfs file being read simultaneously by many dfs clients (in this case all maps) causes the clients to be closed. However on the pig side, loading the sample from each map in the final map reduce job and computing the quantile boundaries and weighted probabilities is inefficient. We should do this computation through a FindQuantiles udf in the same map reduce job which produces the sorted samples. This way lesser data is written to dfs and in the final map reduce job, the weightedRangePartitioner needs to just load the computed information. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-743) to implement clover
[ https://issues.apache.org/jira/browse/PIG-743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-743: --- Attachment: pig-743.patch to implement clover --- Key: PIG-743 URL: https://issues.apache.org/jira/browse/PIG-743 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-743.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (PIG-743) to implement clover
[ https://issues.apache.org/jira/browse/PIG-743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12696430#action_12696430 ] Giridharan Kesavan edited comment on PIG-743 at 4/7/09 1:21 AM: this patch add's clover target's for pig builds. was (Author: gkesavan): this patch add clover targets for pig builds. to implement clover --- Key: PIG-743 URL: https://issues.apache.org/jira/browse/PIG-743 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-743.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-743) to implement clover
[ https://issues.apache.org/jira/browse/PIG-743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated PIG-743: --- Status: Patch Available (was: Open) this patch add clover targets for pig builds. to implement clover --- Key: PIG-743 URL: https://issues.apache.org/jira/browse/PIG-743 Project: Pig Issue Type: Improvement Components: build Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: pig-743.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
