[jira] Commented: (PIG-1389) Implement Pig counter to track number of rows for each input files

2010-06-24 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12882380#action_12882380 ] Arun C Murthy commented on PIG-1389: Can we not just increment the standard MR counters

[jira] Commented: (PIG-1389) Implement Pig counter to track number of rows for each input files

2010-06-24 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12882381#action_12882381 ] Arun C Murthy commented on PIG-1389: How many new counters are we really adding here? I

Re: Consider cleaning up backend code

2010-04-22 Thread Arun C Murthy
+1 Arun On Apr 22, 2010, at 11:35 AM, Richard Ding wrote: Pig has an abstraction layer (interfaces and abstract classes) to support multiple execution engines. After PIG-1053, Hadoop is the only execution engine supported by Pig. I wonder if we should remove this layer of code, and make

Re: Consider cleaning up backend code

2010-04-22 Thread Arun C Murthy
should get rid of the dead localmode code. Can you give an example of how this will simplify the codebase? Is it more than just GenericClass foo = new SpecificClass(), and the associated extra files? -D On Thu, Apr 22, 2010 at 1:25 PM, Arun C Murthy a...@yahoo-inc.com wrote: +1 Arun

Re: Consider cleaning up backend code

2010-04-22 Thread Arun C Murthy
On Apr 22, 2010, at 4:38 PM, Richard Ding wrote: Yes. The abstraction layer I was referring to is src/org/apache/pig/backend/executionengine and src/org/apache/pig/backend/datastorage. Thanks for the clarification. +1 Arun Thanks, -Richard -Original Message- From: Arun C Murthy

[jira] Created: (PIG-1280) Add a pig-script-id to the JobConf of all jobs run in a pig-script

2010-03-05 Thread Arun C Murthy (JIRA)
Components: impl Reporter: Arun C Murthy It would be very useful for tools like gridmix if pig could add a 'pig-script-id' to all Map-Reduce jobs spawned by a single pig-script. Potentially we could use this to re-construct the DAG of jobs in gridmix and so

Re: Private variables are not eco-friendly

2010-02-03 Thread Arun C Murthy
The current model forces people to 'convince' others to open up classes for inheritance at the precise point it is necessary. This is a model which has served, at least, Hadoop very well. So, I think we should not go make every member protected - rather we should open them up one at a

[jira] Commented: (PIG-1218) Use distributed cache to store samples

2010-02-03 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12829305#action_12829305 ] Arun C Murthy commented on PIG-1218: I'd also suggest we increase replication factor

[jira] Commented: (PIG-1062) load-store-redesign branch: change SampleLoader and subclasses to work with new LoadFunc interface

2009-11-16 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12778666#action_12778666 ] Arun C Murthy commented on PIG-1062: bq. It looks like ReduceContext has a getCounter

Re: Proposal to create a branch for contrib project Zebra

2009-08-17 Thread Arun C Murthy
On Aug 17, 2009, at 4:38 PM, Santhosh Srinivasan wrote: Is there any precedence for such proposals? I am not comfortable with extending committer access to contrib teams. I would suggest that Zebra be made a sub-project of Hadoop and have a life of its own. There has been sufficient

Re: Proposal to create a branch for contrib project Zebra

2009-08-17 Thread Arun C Murthy
That leaves us with contrib committers. Can you point to earlier email threads that cover the topic of giving committer access to contrib projects? Specifically, what does it mean to award someone committer privileges to a contrib project, what are the access privileges that come with such

[jira] Commented: (PIG-901) InputSplit (SliceWrapper) created by Pig is big in size due to serialized PigContext

2009-08-04 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12739048#action_12739048 ] Arun C Murthy commented on PIG-901: --- bq. This may require some design changes which we

[jira] Commented: (PIG-901) InputSplit (SliceWrapper) created by Pig is big in size due to serialized PigContext

2009-08-03 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12738740#action_12738740 ] Arun C Murthy commented on PIG-901: --- It would be nice to add a test case which (for now

[jira] Commented: (PIG-878) Pig is returning too many blocks in the InputSplit

2009-07-20 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12733311#action_12733311 ] Arun C Murthy commented on PIG-878: --- bq. Should note also that I didn't add any tests

[jira] Created: (PIG-864) Record graph of execution of Map-Reduce jobs executed by a Pig script

2009-06-25 Thread Arun C Murthy (JIRA)
Reporter: Arun C Murthy It would be useful for offline analysis if Pig were to record the entire graph of Map-Reduce jobs executed by a singe Pig script. For starters a simple 'parent jobid' for each MR job in the graph would be nice. -- This message is automatically generated

Re: [VOTE] Release Pig 0.3.0 (candidate 0)

2009-06-22 Thread Arun C Murthy
On Jun 18, 2009, at 12:30 PM, Olga Natkovich wrote: Hi, I created a candidate build for Pig 0.3.0 release. The main feature of this release is support for multiquery which allows to share computation across multiple queries within the same script. We see significant performance

Re: [VOTE] Release Pig 0.1.1 (candidate 0)

2008-12-02 Thread Arun C Murthy
+1. I downloaded the release, checked the signatures and checksums. All unit test pass. Arun On Nov 25, 2008, at 3:58 PM, Olga Natkovich wrote: Hi, I have created a candidate build for Pig 0.1.1. This release is almost identical to Pig 0.1.0 with a couple of exceptions: (1) It is