[jira] Commented: (PIG-1514) Migrate logical optimization rule: OpLimitOptimizer

2010-08-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901279#action_12901279 ] Daniel Dai commented on PIG-1514: - One minor correction, adding: {code}

Re: [VOTE] Pig to become a top level Apache project

2010-08-23 Thread Alan Gates
With 9 +1 votes and no -1s the vote passes. I will begin a vote on Hadoop general. Alan. On Aug 18, 2010, at 10:34 AM, Alan Gates wrote: Earlier this week I began a discussion on Pig becoming a TLP (http://bit.ly/byD7L8 ). All of the received feedback was positive. So, let's have a

Re: August Pig contributor workshop

2010-08-23 Thread Dmitriy Ryaboy
Olga, We do have another couple of spots. -Dmitriy On Thu, Aug 19, 2010 at 10:28 AM, Olga Natkovich ol...@yahoo-inc.comwrote: Dmitry, Do you have any spots left? Olga -Original Message- From: Russell Jurney [mailto:russell.jur...@gmail.com] Sent: Thursday, August 19, 2010 5:22

[jira] Created: (PIG-1556) Need a clean way to kill Pig jobs.

2010-08-23 Thread Aravind Srinivasan (JIRA)
Need a clean way to kill Pig jobs. -- Key: PIG-1556 URL: https://issues.apache.org/jira/browse/PIG-1556 Project: Pig Issue Type: New Feature Components: tools Affects Versions: 0.7.0

[jira] Updated: (PIG-1178) LogicalPlan and Optimizer are too complex and hard to work with

2010-08-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-1178: Attachment: PIG-1178-7.patch PIG-1178-7.patch switch the flag to use new logical plan by default. It fix

[jira] Updated: (PIG-1178) LogicalPlan and Optimizer are too complex and hard to work with

2010-08-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-1178: Status: Open (was: Patch Available) LogicalPlan and Optimizer are too complex and hard to work with

[jira] Updated: (PIG-1178) LogicalPlan and Optimizer are too complex and hard to work with

2010-08-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-1178: Status: Patch Available (was: Open) LogicalPlan and Optimizer are too complex and hard to work with

[jira] Commented: (PIG-1178) LogicalPlan and Optimizer are too complex and hard to work with

2010-08-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901543#action_12901543 ] Daniel Dai commented on PIG-1178: - PIG-1178-7.patch committed. LogicalPlan and Optimizer

[jira] Updated: (PIG-506) Does pig need a NATIVE keyword?

2010-08-23 Thread Thejas M Nair (JIRA)
[ https://issues.apache.org/jira/browse/PIG-506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated PIG-506: -- Attachment: PIG-506.patch New patch address my comments. test-patch results - [exec] -1 overall.

[jira] Commented: (PIG-1555) [piggybank] add CSV Loader

2010-08-23 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901556#action_12901556 ] Alan Gates commented on PIG-1555: - +1 If you have a chance sometime I'd be curious to learn

[jira] Commented: (PIG-1508) Make 'docs' target (forrest) work with Java 1.6

2010-08-23 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901559#action_12901559 ] Alan Gates commented on PIG-1508: - Alright, I'll get this checked in before we branch for 0.8

[jira] Updated: (PIG-908) Need a way to correlate MR jobs with Pig statements

2010-08-23 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-908: --- With Pig 0.8.0 we print a summary of the execution that contains (among other things) how aliases mapped

[jira] Updated: (PIG-1488) Make HDFS temp dir configurable

2010-08-23 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1488: Release Note: Pig stores intermediate data generated between MR jobs in a temp location on HDFS. In

[jira] Updated: (PIG-1505) support jars and scripts in dfs

2010-08-23 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Ding updated PIG-1505: -- Release Note: Pig now supports running scripts and registering jars that are stored in HDFS, Amazon S3,

[jira] Updated: (PIG-1484) BinStorage should support comma seperated path

2010-08-23 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1484: Release Note: In Pig 0.7.0 only a single location is supported as input to BinStorage. (This

[jira] Updated: (PIG-1447) Tune memory usage of InternalCachedBag

2010-08-23 Thread Thejas M Nair (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated PIG-1447: --- Status: Patch Available (was: Open) Patch for increasing default value to 20%. No new test cases as

[jira] Created: (PIG-1557) couple of issue mapping aliases to jobs

2010-08-23 Thread Olga Natkovich (JIRA)
couple of issue mapping aliases to jobs --- Key: PIG-1557 URL: https://issues.apache.org/jira/browse/PIG-1557 Project: Pig Issue Type: Bug Affects Versions: 0.8.0 Reporter: Olga Natkovich

RE: August Pig contributor workshop

2010-08-23 Thread Olga Natkovich
Ok, thanks Dmitry we have at least one more person coming with us. Olga -Original Message- From: Dmitriy Ryaboy [mailto:dvrya...@gmail.com] Sent: Monday, August 23, 2010 10:02 AM To: pig-dev@hadoop.apache.org Subject: Re: August Pig contributor workshop Olga, We do have another couple

[jira] Commented: (PIG-1447) Tune memory usage of InternalCachedBag

2010-08-23 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901576#action_12901576 ] Olga Natkovich commented on PIG-1447: - This is probably the smallest patch I have

[jira] Commented: (PIG-1354) UDFs for dynamic invocation of simple Java methods

2010-08-23 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901577#action_12901577 ] Olga Natkovich commented on PIG-1354: - Dmitry, Could you add release notes on how to use

[jira] Updated: (PIG-1447) Tune memory usage of InternalCachedBag

2010-08-23 Thread Thejas M Nair (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated PIG-1447: --- Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed

[jira] Commented: (PIG-1354) UDFs for dynamic invocation of simple Java methods

2010-08-23 Thread Dmitriy V. Ryaboy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901584#action_12901584 ] Dmitriy V. Ryaboy commented on PIG-1354: Olga, There is a follow-up ticket here:

[jira] Commented: (PIG-1354) UDFs for dynamic invocation of simple Java methods

2010-08-23 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901585#action_12901585 ] Olga Natkovich commented on PIG-1354: - Sounds good, Dmitry. Richard will review and

[jira] Commented: (PIG-1508) Make 'docs' target (forrest) work with Java 1.6

2010-08-23 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901586#action_12901586 ] Alan Gates commented on PIG-1508: - I can't figure out a way to test test-patch.sh without

[jira] Commented: (PIG-1311) Pig interfaces should be clearly classified in terms of scope and stability

2010-08-23 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901587#action_12901587 ] Olga Natkovich commented on PIG-1311: - +1, please, commit Pig interfaces should be

[jira] Created: (PIG-1558) build.xml for site directory does not work

2010-08-23 Thread Alan Gates (JIRA)
build.xml for site directory does not work -- Key: PIG-1558 URL: https://issues.apache.org/jira/browse/PIG-1558 Project: Pig Issue Type: Bug Components: build Affects Versions: 0.8.0

[jira] Commented: (PIG-1552) Nested describe failed when the alias is not referred in the first foreach inner plan

2010-08-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901593#action_12901593 ] Daniel Dai commented on PIG-1552: - Unit test pass. test-patch result: [exec] +1

[jira] Updated: (PIG-1552) Nested describe failed when the alias is not referred in the first foreach inner plan

2010-08-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-1552: Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed Patch

[jira] Commented: (PIG-1518) multi file input format for loaders

2010-08-23 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901600#action_12901600 ] Richard Ding commented on PIG-1518: --- +1. The patch looks good. A few of minor points: *

[jira] Updated: (PIG-1558) build.xml for site directory does not work

2010-08-23 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-1558: Attachment: PIG-1558.patch Attached patch makes it so that the ant invocation requires the user to specify

[jira] Created: (PIG-1559) Several things stated in Pig philosophy page are out of date

2010-08-23 Thread Alan Gates (JIRA)
Several things stated in Pig philosophy page are out of date Key: PIG-1559 URL: https://issues.apache.org/jira/browse/PIG-1559 Project: Pig Issue Type: Bug Components:

[jira] Commented: (PIG-1558) build.xml for site directory does not work

2010-08-23 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901612#action_12901612 ] Olga Natkovich commented on PIG-1558: - +1 build.xml for site directory does not work

[jira] Commented: (PIG-1508) Make 'docs' target (forrest) work with Java 1.6

2010-08-23 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901617#action_12901617 ] Alan Gates commented on PIG-1508: - I'm guessing the contrib failures are just because Hudson

Pig optimizer

2010-08-23 Thread Renato Marroquín Mogrovejo
Hey everyone, I was wondering if anybody has any references or suggestion on how to learn about Pig's optimizer besides the source code or Pig's paper. Thanks in advance. Renato M.

[jira] Updated: (PIG-1510) Add `deepCopy` for LogicalExpressions

2010-08-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-1510: Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed Patch

[jira] Updated: (PIG-1559) Several things stated in Pig philosophy page are out of date

2010-08-23 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-1559: Attachment: PIG-1559.patch Several things stated in Pig philosophy page are out of date

[jira] Updated: (PIG-1559) Several things stated in Pig philosophy page are out of date

2010-08-23 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-1559: Status: Patch Available (was: Open) Several things stated in Pig philosophy page are out of date

Re: Pig optimizer

2010-08-23 Thread Daniel Dai
Hi, Renato, There is a description of optimization rule in Pig Latin reference menu: http://hadoop.apache.org/pig/docs/r0.7.0/piglatin_ref1.html#Optimization+Rules. Is that enough? Daniel Renato Marroquín Mogrovejo wrote: Hey everyone, I was wondering if anybody has any references or

is Hudson awol?

2010-08-23 Thread Dmitriy Ryaboy
Haven't heard anything from Hudson in a while... -D

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch The add method if PigSplit is removed. The debug code is left to facilitate future

[jira] Commented: (PIG-1551) Improve dynamic invokers to deal with no-arg methods and array parameters

2010-08-23 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901656#action_12901656 ] Richard Ding commented on PIG-1551: --- In Invoker.java, there is a typo: {code} private

Re: split operator

2010-08-23 Thread Gang Luo
Hi Daniel, This is a question long ago, but I suddenly come up with some more thoughts on this. In a query as simple as this: A = LOAD 'input'; B = FILTER A BY $1 == 1; C = COGROUP A BY $0, B BY $0; the optimizer will insert a split operator to reuse A. According to the source code, a

[jira] Created: (PIG-1560) Build target 'checkstyle' fails

2010-08-23 Thread Richard Ding (JIRA)
Build target 'checkstyle' fails --- Key: PIG-1560 URL: https://issues.apache.org/jira/browse/PIG-1560 Project: Pig Issue Type: Bug Affects Versions: 0.8.0 Reporter: Richard Ding

[jira] Updated: (PIG-1560) Build target 'checkstyle' fails

2010-08-23 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Ding updated PIG-1560: -- Description: Stack trace: {code} /trunk/build.xml:894: java.lang.NoClassDefFoundError:

Re: split operator

2010-08-23 Thread Daniel Dai
Hi, Gang, Yes, that's what MultiQueryOptimizer address. After splitting, we split the script into smaller combinable pieces, and MultiQueryOptimizer will combine as much splitter and splittees into the same map-reduce job. So after SplitInserter, you might see more jobs, but you will end up with

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Attachment: PIG-1518.patch Fix a typo; rebase on the latest trunk. multi file input format for loaders

[jira] Updated: (PIG-1515) Migrate logical optimization rule: PushDownForeachFlatten

2010-08-23 Thread Xuefu Zhang (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated PIG-1515: - Status: Patch Available (was: Open) Migrate logical optimization rule: PushDownForeachFlatten

[jira] Updated: (PIG-1557) couple of issue mapping aliases to jobs

2010-08-23 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Ding updated PIG-1557: -- Attachment: PIG-1557.patch The alias for load statement is missing. Add load alias to the alias list.

[jira] Updated: (PIG-1557) couple of issue mapping aliases to jobs

2010-08-23 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Ding updated PIG-1557: -- Fix Version/s: 0.8.0 couple of issue mapping aliases to jobs ---

[jira] Updated: (PIG-1518) multi file input format for loaders

2010-08-23 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1518: -- Status: Patch Available (was: Open) Release Note: Feature: combine splits of sizes smaller than the

[jira] Created: (PIG-1561) XMLLoader in Piggybank does not support bz2 or gzip compressed XML files

2010-08-23 Thread Viraj Bhat (JIRA)
XMLLoader in Piggybank does not support bz2 or gzip compressed XML files Key: PIG-1561 URL: https://issues.apache.org/jira/browse/PIG-1561 Project: Pig Issue Type: Bug

[jira] Commented: (PIG-1555) [piggybank] add CSV Loader

2010-08-23 Thread Dmitriy V. Ryaboy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12901697#action_12901697 ] Dmitriy V. Ryaboy commented on PIG-1555: Alan, The differences I observe when running