Re: Need help

2013-11-27 Thread Pradeep Gollakota
This question belongs on the user list. The dev list is meant for Pig developers to discuss issues related to the development of Pig. I’ve forwarded this to the user list. It also helps tremendously if you format your data and scripts nicely as they’re much easier to read and understand. I use a ch

Need help

2013-11-27 Thread Haider
Hi Daniel I need help so badly , I hope you would understand my situation The use case is, I have one folder which has multiple XML files and I need to write a PIG script which recursively parse all the files and generate one flat file. The XML looks like this and each XML file has differe

[jira] [Updated] (PIG-3590) remove PartitionFilterOptimizer from trunk

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3590: Fix Version/s: 0.13.0 > remove PartitionFilterOptimizer from trunk >

[jira] [Commented] (PIG-3590) remove PartitionFilterOptimizer from trunk

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834439#comment-13834439 ] Aniket Mokashi commented on PIG-3590: - Committed to trunk. Thanks for reviewing [~cheols

[jira] [Updated] (PIG-3590) remove PartitionFilterOptimizer from trunk

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3590: Resolution: Fixed Status: Resolved (was: Patch Available) > remove PartitionFilterOptimizer

[jira] Subscription: PIG patch available

2013-11-27 Thread jira
Issue Subscription Filter: PIG patch available (9 issues) Subscriber: pigdaily Key Summary PIG-3592Should not try to create success file for non-fs schemes like hbase https://issues.apache.org/jira/browse/PIG-3592 PIG-3590remove PartitionFilterOptimizer from trunk

[jira] [Updated] (PIG-3590) remove PartitionFilterOptimizer from trunk

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3590: Attachment: PIG-3590.patch > remove PartitionFilterOptimizer from trunk > ---

[jira] [Updated] (PIG-3590) remove PartitionFilterOptimizer from trunk

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3590: Attachment: (was: PIG-3590.patch) > remove PartitionFilterOptimizer from trunk >

[jira] [Updated] (PIG-3591) Refactor POPackage to separate MR specific code from packaging

2013-11-27 Thread Mark Wagner (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated PIG-3591: - Attachment: PIG-3591.2.patch > Refactor POPackage to separate MR specific code from packaging > ---

[jira] [Updated] (PIG-3527) Allow PigProcessor to handle multiple inputs

2013-11-27 Thread Mark Wagner (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated PIG-3527: - Attachment: PIG-3527.2.patch Update with the POPackage refactoring. This patch depends on the one in PIG-

[jira] [Updated] (PIG-3595) Port Package refactoring to Tez branch

2013-11-27 Thread Mark Wagner (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated PIG-3595: - Attachment: PIG-3595.1.patch Here's the first backport for the tez branch. > Port Package refactoring to T

Re: Review Request 15194: Support multiple inputs for PigProcessor

2013-11-27 Thread Mark Wagner
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15194/ --- (Updated Nov. 28, 2013, 12:41 a.m.) Review request for pig, Cheolsoo Park, Dani

Re: Review Request 15881: PIG-3591: Refactor POPackage

2013-11-27 Thread Mark Wagner
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15881/ --- (Updated Nov. 28, 2013, 12:39 a.m.) Review request for pig and Cheolsoo Park.

[jira] [Commented] (PIG-3590) remove PartitionFilterOptimizer from trunk

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834403#comment-13834403 ] Aniket Mokashi commented on PIG-3590: - Yes, I missed that one. Will remove it. Thanks!

[jira] [Created] (PIG-3595) Port Package refactoring to Tez branch

2013-11-27 Thread Mark Wagner (JIRA)
Mark Wagner created PIG-3595: Summary: Port Package refactoring to Tez branch Key: PIG-3595 URL: https://issues.apache.org/jira/browse/PIG-3595 Project: Pig Issue Type: Sub-task Repor

[jira] [Commented] (PIG-3590) remove PartitionFilterOptimizer from trunk

2013-11-27 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834398#comment-13834398 ] Cheolsoo Park commented on PIG-3590: [~aniket486], do you mind removing the comments in

[jira] [Updated] (PIG-3590) remove PartitionFilterOptimizer from trunk

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3590: Attachment: PIG-3590.patch > remove PartitionFilterOptimizer from trunk > ---

[jira] [Assigned] (PIG-3590) remove PartitionFilterOptimizer from trunk

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi reassigned PIG-3590: --- Assignee: Aniket Mokashi > remove PartitionFilterOptimizer from trunk > ---

[jira] [Updated] (PIG-3590) remove PartitionFilterOptimizer from trunk

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3590: Status: Patch Available (was: Open) > remove PartitionFilterOptimizer from trunk > -

[jira] [Resolved] (PIG-3566) Cannot set useMatches of REGEX_EXTRACT_ALL and REGEX_EXTRACT

2013-11-27 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park resolved PIG-3566. Resolution: Fixed Fix Version/s: 0.13.0 Committed to trunk. Thank you Nazih! > Cannot set use

[jira] [Commented] (PIG-3594) Pig with HCatLoader partition filter does not push down valid conditions

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834349#comment-13834349 ] Aniket Mokashi commented on PIG-3594: - This is fixed in 0.12.1 with https://issues.apach

[jira] [Updated] (PIG-3594) Pig with HCatLoader partition filter does not push down valid conditions

2013-11-27 Thread Mona Chitnis (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mona Chitnis updated PIG-3594: -- Summary: Pig with HCatLoader partition filter does not push down valid conditions (was: Pig with HCatLoa

[jira] [Created] (PIG-3594) Pig with HCatLoader scans all partitions

2013-11-27 Thread Mona Chitnis (JIRA)
Mona Chitnis created PIG-3594: - Summary: Pig with HCatLoader scans all partitions Key: PIG-3594 URL: https://issues.apache.org/jira/browse/PIG-3594 Project: Pig Issue Type: Bug Affects Versio

[jira] [Commented] (PIG-3592) Should not try to create success file for non-fs schemes like hbase

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834201#comment-13834201 ] Aniket Mokashi commented on PIG-3592: - +1 > Should not try to create success file for n

[jira] [Updated] (PIG-3576) NPE due to PIG-3549 when job never gets submitted

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3576: Fix Version/s: 0.12.1 > NPE due to PIG-3549 when job never gets submitted > -

[jira] [Commented] (PIG-3576) NPE due to PIG-3549 when job never gets submitted

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834197#comment-13834197 ] Aniket Mokashi commented on PIG-3576: - Thanks [~cheolsoo], I've attached the new patch.

[jira] [Updated] (PIG-3576) NPE due to PIG-3549 when job never gets submitted

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3576: Attachment: PIG-3576-1.patch > NPE due to PIG-3549 when job never gets submitted > --

[jira] [Updated] (PIG-3592) Should not try to create success file for non-fs schemes like hbase

2013-11-27 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3592: Fix Version/s: 0.13.0 Status: Patch Available (was: Open) > Should not try to cre

[jira] [Updated] (PIG-3592) Should not try to create success file for non-fs schemes like hbase

2013-11-27 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3592: Attachment: PIG-3592-branch12-1.patch PIG-3592-1.patch Attached patches for t

[jira] [Updated] (PIG-2132) Piggybank: MIN and MAX functions should ignore nulls

2013-11-27 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-2132: --- Resolution: Fixed Assignee: Rekha Joshi Status: Resolved (was: Patch Available) Committe

[jira] [Updated] (PIG-2132) Piggybank: MIN and MAX functions should ignore nulls

2013-11-27 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-2132: --- Fix Version/s: 0.13.0 > Piggybank: MIN and MAX functions should ignore nulls >

[jira] [Commented] (PIG-3593) Import jython standard module fail on cluster

2013-11-27 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834157#comment-13834157 ] Rohini Palaniswamy commented on PIG-3593: - bq. I am not sure whether we fixed issue

[jira] [Commented] (PIG-3576) NPE due to PIG-3549 when job never gets submitted

2013-11-27 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834151#comment-13834151 ] Cheolsoo Park commented on PIG-3576: [~aniket486], no sir. Go ahead. > NPE due to PIG-3

[jira] [Commented] (PIG-3593) Import jython standard module fail on cluster

2013-11-27 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834149#comment-13834149 ] Rohini Palaniswamy commented on PIG-3593: - +1. Just a minor comment. You don't have

[jira] [Updated] (PIG-3593) Import jython standard module fail on cluster

2013-11-27 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-3593: Attachment: PIG-3593-1.patch asm does not complain if we ship jython.jar as a single unit and put in distrib

[jira] [Created] (PIG-3593) Import jython standard module fail on cluster

2013-11-27 Thread Daniel Dai (JIRA)
Daniel Dai created PIG-3593: --- Summary: Import jython standard module fail on cluster Key: PIG-3593 URL: https://issues.apache.org/jira/browse/PIG-3593 Project: Pig Issue Type: Bug Compone

[jira] [Commented] (PIG-3566) Cannot set useMatches of REGEX_EXTRACT_ALL and REGEX_EXTRACT

2013-11-27 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834117#comment-13834117 ] Nezih Yigitbasi commented on PIG-3566: -- Cheolsoo, I have also updated the TestBuiltin c

[jira] [Updated] (PIG-3566) Cannot set useMatches of REGEX_EXTRACT_ALL and REGEX_EXTRACT

2013-11-27 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nezih Yigitbasi updated PIG-3566: - Attachment: PIG-3566.1.patch Fixed the TestBuiltin test class. > Cannot set useMatches of REGEX_EX

[jira] [Commented] (PIG-3576) NPE due to PIG-3549 when job never gets submitted

2013-11-27 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834104#comment-13834104 ] Aniket Mokashi commented on PIG-3576: - [~cheolsoo], we need this on pig-12 branch. Any o

[jira] [Updated] (PIG-2095) pig start script doesn't collect libs properly when "hadoop" is part of PIG_HOME

2013-11-27 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-2095: --- Resolution: Won't Fix Status: Resolved (was: Patch Available) [~rekhajoshm], thank you for the

[jira] [Updated] (PIG-2132) Piggybank: MIN and MAX functions should ignore nulls

2013-11-27 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-2132: --- Attachment: PIG-2132_2.patch Here is the patch after clean up. I will commit this. > Piggybank: MIN an

[jira] [Commented] (PIG-2132) Piggybank: MIN and MAX functions should ignore nulls

2013-11-27 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834075#comment-13834075 ] Cheolsoo Park commented on PIG-2132: [~rekhajoshm], thank you for the patch. But your pa

[jira] [Commented] (PIG-3587) add functionality for rolling over dates

2013-11-27 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834007#comment-13834007 ] Cheolsoo Park commented on PIG-3587: [~rekhajoshm], thank you very much for the patch.

[jira] [Created] (PIG-3592) Should not try to create success file for non-fs schemes like hbase

2013-11-27 Thread Rohini Palaniswamy (JIRA)
Rohini Palaniswamy created PIG-3592: --- Summary: Should not try to create success file for non-fs schemes like hbase Key: PIG-3592 URL: https://issues.apache.org/jira/browse/PIG-3592 Project: Pig

[jira] [Updated] (PIG-3591) Refactor POPackage to separate MR specific code from packaging

2013-11-27 Thread Mark Wagner (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated PIG-3591: - Attachment: PIG-3591.1.patch Separate "packaging" logic from "shuffle handling" logic. This moves the pack

Review Request 15881: PIG-3591: Refactor POPackage

2013-11-27 Thread Mark Wagner
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15881/ --- Review request for pig and Cheolsoo Park. Bugs: PIG-3591 https://issues.apa

[jira] [Created] (PIG-3591) Refactor POPackage to separate MR specific code from packaging

2013-11-27 Thread Mark Wagner (JIRA)
Mark Wagner created PIG-3591: Summary: Refactor POPackage to separate MR specific code from packaging Key: PIG-3591 URL: https://issues.apache.org/jira/browse/PIG-3591 Project: Pig Issue Type: B