[jira] Commented: (PIG-1249) Safe-guards against misconfigured Pig scripts without PARALLEL keyword

2010-07-12 Thread Ashutosh Chauhan (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887283#action_12887283 ] Ashutosh Chauhan commented on PIG-1249: --- Map-reduce framework has a jira related to

[jira] Updated: (PIG-928) UDFs in scripting languages

2010-07-12 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-928: --- Status: Patch Available (was: Open) UDFs in scripting languages ---

[jira] Updated: (PIG-928) UDFs in scripting languages

2010-07-12 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-928: --- Attachment: (was: RegisterPythonUDF2.patch) UDFs in scripting languages

[jira] Commented: (PIG-1490) Make Pig storers work with remote HDFS in secure mode

2010-07-12 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887424#action_12887424 ] Daniel Dai commented on PIG-1490: - +1 Make Pig storers work with remote HDFS in secure mode

[jira] Commented: (PIG-1494) PIG Logical Optimization: Use CNF in PushUpFilter

2010-07-12 Thread Swati Jain (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887425#action_12887425 ] Swati Jain commented on PIG-1494: - Reply from Yan Zhou: The filter logic split problem can

[jira] Commented: (PIG-1295) Binary comparator for secondary sort

2010-07-12 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887420#action_12887420 ] Daniel Dai commented on PIG-1295: - More clarification for custom Tuple. There two cases for

[jira] Updated: (PIG-928) UDFs in scripting languages

2010-07-12 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-928: --- Status: Open (was: Patch Available) UDFs in scripting languages ---

[jira] Created: (PIG-1494) PIG Logical Optimization: Use CNF in PushUpFilter

2010-07-12 Thread Swati Jain (JIRA)
PIG Logical Optimization: Use CNF in PushUpFilter - Key: PIG-1494 URL: https://issues.apache.org/jira/browse/PIG-1494 Project: Pig Issue Type: Improvement Components: impl

[jira] Updated: (PIG-1490) Make Pig storers work with remote HDFS in secure mode

2010-07-12 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Ding updated PIG-1490: -- Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Release Note: Committed

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Yan Zhou
Yes, I already implemented the NOT push down upfront, so you do not need to do that. The support of CNF will probably be the most difficulty part. But as I mentioned last time, you should compare the cost after the trimming CNF to get the post-split filtering logic. Given the complexity of

[jira] Commented: (PIG-1472) Optimize serialization/deserialization between Map and Reduce and between MR jobs

2010-07-12 Thread Thejas M Nair (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887441#action_12887441 ] Thejas M Nair commented on PIG-1472: bq. 1. The following code are never used in

[jira] Updated: (PIG-1472) Optimize serialization/deserialization between Map and Reduce and between MR jobs

2010-07-12 Thread Thejas M Nair (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated PIG-1472: --- Attachment: PIG-1472.4.patch Removed unused static constants from InterStorage and BinStorage ,

[jira] Commented: (PIG-1436) Print number of records outputted at each step of a Pig script

2010-07-12 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887446#action_12887446 ] Richard Ding commented on PIG-1436: --- Russell, PIG-1478 implemented a callback mechanism

[jira] Resolved: (PIG-884) Have a way to export RulePlan and other kinds of OperatorPlan to common representaiton (dot?) and import from dot to RulePlan

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-884. Resolution: Fixed dot notation for explain was used as part of Pig 0.3.0 work. Have a way to export

[jira] Resolved: (PIG-886) clone should be updated in LogicalOperators to include cloning of projection map information and any other information used by LogicalOptimizer

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-886. Resolution: Fixed This is no longer relevant with the optimizer re-work clone should be updated in

[jira] Updated: (PIG-900) ORDER BY syntax wrt parentheses is somewhat different than GROUP BY and FILTER BY

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-900: --- Fix Version/s: 0.9.0 ORDER BY syntax wrt parentheses is somewhat different than GROUP BY and FILTER

[jira] Updated: (PIG-902) Allow schema matching for UDF with variable length arguments

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-902: --- Fix Version/s: 0.9.0 Allow schema matching for UDF with variable length arguments

[jira] Updated: (PIG-903) ILLUSTRATE fails on 'Distinct' operator

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-903: --- Fix Version/s: 0.9.0 ILLUSTRATE fails on 'Distinct' operator ---

[jira] Resolved: (PIG-898) TextDataParser does not handle delimiters from one complex type in another

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-898. Fix Version/s: 0.7.0 Resolution: Fixed This has been addressed as part of 613 TextDataParser

Re: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Swati Jain
I was wondering if you are not going to check in your patch soon then it would be great if you could share it with me. I believe I might be able to reuse some of your (utility) functionality directly or get some ideas. About your cost-benefit question: 1) I will control the complexity of CNF

[jira] Commented: (PIG-914) Change the PIG hbase interface to use bytes along with strings

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887479#action_12887479 ] Olga Natkovich commented on PIG-914: ALex, are you still planning to work on this?

[jira] Commented: (PIG-916) Change the pig hbase interface to get more than one row at a time when scanning

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887480#action_12887480 ] Olga Natkovich commented on PIG-916: Alex, are you still planning to work on this?

[jira] Commented: (PIG-909) Allow Pig executable to use hadoop jars not bundled with pig

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887476#action_12887476 ] Olga Natkovich commented on PIG-909: Did this actually get checked in? Should this be

[jira] Updated: (PIG-932) Required fields projection in Loader: nested fields in bag/tuple, map key lookup more than two levels

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-932: --- Fix Version/s: 0.8.0 Required fields projection in Loader: nested fields in bag/tuple, map key

[jira] Updated: (PIG-931) Samples Syntax Error in Pig UDF Manual

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-931: --- Assignee: Corinne Chandel Fix Version/s: 0.8.0 Samples Syntax Error in Pig UDF Manual

[jira] Updated: (PIG-930) merge join should handle compressed bz2 sorted files

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-930: --- Fix Version/s: 0.8.0 Likely, this is no longer an issue in 0.7.0. Need to verify and add unit tests

[jira] Updated: (PIG-932) Required fields projection in Loader: nested fields in bag/tuple, map key lookup more than two levels

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-932: --- Assignee: Daniel Dai Possible work for 0.8.0. Need to see if we have time Required fields projection

[jira] Updated: (PIG-947) Parsing Bags by PigStorage is not handled correctly if whitespace before start of tuple.

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-947: --- Fix Version/s: 0.8.0 Parsing Bags by PigStorage is not handled correctly if whitespace before start

[jira] Updated: (PIG-969) Default constructor of UDF gets called for UDF with parameterised constructor , if the udf has a getArgToFuncMapping function defined

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-969: --- Fix Version/s: 0.9.0 Description: This issue is discussed in

[jira] Resolved: (PIG-1182) Pig reference manual does not mention syntax for comments

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-1182. - Resolution: Fixed Closing. If we do want to do an comprehansive index, please, create a separate

[jira] Updated: (PIG-999) sorting on map-value fails if map-value is not of bytearray type

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-999: --- Fix Version/s: 0.9.0 sorting on map-value fails if map-value is not of bytearray type

[jira] Updated: (PIG-998) revisit frontend logic and pig-latin semantics

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-998: --- Fix Version/s: 0.9.0 revisit frontend logic and pig-latin semantics

[jira] Resolved: (PIG-967) Proposal for adding a metadata interface to Pig

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-967. Resolution: Won't Fix This is an obsolete proposal Proposal for adding a metadata interface to Pig

[jira] Updated: (PIG-1065) In-determinate behaviour of Union when there are 2 non-matching schema's

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1065: Fix Version/s: 0.9.0 In-determinate behaviour of Union when there are 2 non-matching schema's

[jira] Updated: (PIG-1066) ILLUSTRATE called after DESCRIBE results in Grunt: ERROR 2999: Unexpected internal error. null

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1066: Fix Version/s: 0.9.0 ILLUSTRATE called after DESCRIBE results in Grunt: ERROR 2999: Unexpected

[jira] Resolved: (PIG-1056) table can not be loaded after store

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-1056. - Resolution: Invalid The script is invalid and that's why you see the error table can not be

[jira] Updated: (PIG-1092) Pig Latin Parser fails to recognize \n as a whitespace

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1092: Fix Version/s: 0.9.0 Pig Latin Parser fails to recognize \n as a whitespace

[jira] Updated: (PIG-1112) FLATTEN eliminates the alias

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1112: Fix Version/s: 0.9.0 FLATTEN eliminates the alias

[jira] Updated: (PIG-1017) Converts strings to text in Pig

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1017: Fix Version/s: 0.9.0 Converts strings to text in Pig ---

[jira] Updated: (PIG-1152) bincond operator throws parser error

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1152: Fix Version/s: 0.9.0 bincond operator throws parser error

[jira] Updated: (PIG-1178) LogicalPlan and Optimizer are too complex and hard to work with

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1178: Fix Version/s: 0.8.0 LogicalPlan and Optimizer are too complex and hard to work with

[jira] Resolved: (PIG-1235) OptimizerException: Problem while rebuilding projection map or schema in logical optimizer

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-1235. - Resolution: Won't Fix This is not relevant with new optimizer OptimizerException: Problem while

[jira] Updated: (PIG-1247) Error Number makes it hard to debug: ERROR 2999: Unexpected internal error. org.apache.pig.backend.datastorage.DataStorageException cannot be cast to java.lang.Error

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1247: Fix Version/s: 0.9.0 Error Number makes it hard to debug: ERROR 2999: Unexpected internal error.

[jira] Updated: (PIG-1277) Pig should give error message when cogroup on tuple keys of different inner type

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1277: Fix Version/s: 0.9.0 Pig should give error message when cogroup on tuple keys of different inner

[jira] Updated: (PIG-1319) New logical optimization rules

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1319: Fix Version/s: 0.8.0 New logical optimization rules --

[jira] Resolved: (PIG-1328) pigtest ant target fails pigtrunk builds

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-1328. - Resolution: Fixed I believe all tests are running now. Please, re-open and clarify if this is

[jira] Updated: (PIG-1188) Padding nulls to the input tuple according to input schema

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1188: Fix Version/s: 0.9.0 Padding nulls to the input tuple according to input schema

[jira] Updated: (PIG-1452) to remove hadoop20.jar from lib and use hadoop from the apache maven repo.

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1452: Fix Version/s: 0.8.0 to remove hadoop20.jar from lib and use hadoop from the apache maven repo.

[jira] Updated: (PIG-1387) Syntactical Sugar for PIG-1385

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1387: Fix Version/s: 0.9.0 Syntactical Sugar for PIG-1385 --

[jira] Updated: (PIG-1341) BinStorage cannot convert DataByteArray to Chararray and results in FIELD_DISCARDED_TYPE_CONVERSION_FAILED

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1341: Fix Version/s: 0.9.0 BinStorage cannot convert DataByteArray to Chararray and results in

[jira] Updated: (PIG-1358) [piggybank] String functions should handle exceptions in a consistent manner

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1358: Fix Version/s: 0.9.0 [piggybank] String functions should handle exceptions in a consistent manner

[jira] Updated: (PIG-1399) Logical Optimizer: Expression optimizor rule

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1399: Fix Version/s: 0.8.0 Logical Optimizer: Expression optimizor rule

[jira] Updated: (PIG-1459) Need a standard way to communicate the requested fields between front and back end for loaders

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1459: Fix Version/s: 0.9.0 Need a standard way to communicate the requested fields between front and

[jira] Updated: (PIG-1477) Syntax error in tutorial Pig Script 1: Query Phrase Popularity (ORDER operator)

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1477: Assignee: Corinne Chandel Syntax error in tutorial Pig Script 1: Query Phrase Popularity (ORDER

[jira] Resolved: (PIG-1436) Print number of records outputted at each step of a Pig script

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-1436. - Resolution: Duplicate This looks like duplicate of PIG-1478. Please, re-open if this is not the

[jira] Updated: (PIG-1465) Filter inside foreach is broken

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1465: Fix Version/s: 0.8.0 Filter inside foreach is broken ---

[jira] Resolved: (PIG-1470) map/red jobs fail using G1 GC (Couldn't find heap)

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-1470. - Resolution: Won't Fix Closing since there is no fix in Pig required. Feel gree to continue the

[jira] Updated: (PIG-1492) DefaultTuple and DefaultMemory understimate their memory footprint

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1492: Assignee: Thejas M Nair Fix Version/s: 0.8.0 DefaultTuple and DefaultMemory understimate

[jira] Assigned: (PIG-523) help in grunt should show all commands

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich reassigned PIG-523: -- Assignee: Olga Natkovich help in grunt should show all commands

[jira] Assigned: (PIG-347) Pig (help) Commands

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich reassigned PIG-347: -- Assignee: Olga Natkovich Pig (help) Commands --- Key: PIG-347

[jira] Commented: (PIG-1494) PIG Logical Optimization: Use CNF in PushUpFilter

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887538#action_12887538 ] Olga Natkovich commented on PIG-1494: - Swati, I am assigning it to you since I am

[jira] Assigned: (PIG-1494) PIG Logical Optimization: Use CNF in PushUpFilter

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich reassigned PIG-1494: --- Assignee: Swati Jain PIG Logical Optimization: Use CNF in PushUpFilter

[jira] Commented: (PIG-1472) Optimize serialization/deserialization between Map and Reduce and between MR jobs

2010-07-12 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887545#action_12887545 ] Daniel Dai commented on PIG-1472: - +1 for commit. Optimize serialization/deserialization

[jira] Commented: (PIG-1430) ISODateTime - DateTime: DateTime UDFs Should Also Support int/second Unix Times in All Operations

2010-07-12 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887546#action_12887546 ] Alan Gates commented on PIG-1430: - I think it's fine to start with just putting conversion

[jira] Created: (PIG-1495) Add -q command line option to set queue name for Pig jobs from command line

2010-07-12 Thread Russell Jurney (JIRA)
Add -q command line option to set queue name for Pig jobs from command line --- Key: PIG-1495 URL: https://issues.apache.org/jira/browse/PIG-1495 Project: Pig Issue

[jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1321: Fix Version/s: 0.8.0 Logical Optimizer: Merge cascading foreach

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Yan Zhou
Hopefully by this week. I'm still in the debugging phase of the work. While you are welcome to reuse some of my algorithms, I doubt you can reuse the code as much as you want. It's basically for my DNF use. You might need to factor out some general codes which you can find reusable. I fully

[jira] Updated: (PIG-1472) Optimize serialization/deserialization between Map and Reduce and between MR jobs

2010-07-12 Thread Thejas M Nair (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated PIG-1472: --- Status: Resolved (was: Patch Available) Resolution: Fixed Patch committed to trunk.

[jira] Updated: (PIG-1495) Add -q command line option to set queue name for Pig jobs from command line

2010-07-12 Thread Russell Jurney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Russell Jurney updated PIG-1495: Status: Patch Available (was: Open) Add -q command line option to set queue name for Pig jobs from

[jira] Resolved: (PIG-1368) Utf8StorageConvertor's bytesToTuple and bytesToBag methods need to be tightened for corner cases

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-1368. - Resolution: Duplicate This will be addressed as part of PIG-1271 Utf8StorageConvertor's

[jira] Assigned: (PIG-1466) Improve log messages for memory usage

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich reassigned PIG-1466: --- Assignee: Thejas M Nair Thejas, can you update the messages since you are already looking at

[jira] Updated: (PIG-1495) Add -q command line option to set queue name for Pig jobs from command line

2010-07-12 Thread Russell Jurney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Russell Jurney updated PIG-1495: Status: Open (was: Patch Available) Add -q command line option to set queue name for Pig jobs from

PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Swati Jain
Yan, What I meant in my last email was that scenario 2 optimizations would lead to more opportunities for scenario 1 kind of optimizations. Consider the conjunct list [C1;C2;C3] as the source of a JOIN. (a) Suppose none of these are computable on a join input, in this case we retain the

[jira] Commented: (PIG-1478) Add progress notification listener to PigRunner API

2010-07-12 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887578#action_12887578 ] Alan Gates commented on PIG-1478: - I don't understand the difference between

[jira] Assigned: (PIG-1460) UDF manual and javadocs should make clear how to use RequiredFieldList

2010-07-12 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich reassigned PIG-1460: --- Assignee: Pradeep Kamath Pradeep, could you provide the information needed and also update the

[jira] Commented: (PIG-1373) We need to add jdiff output to docs on the website

2010-07-12 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887582#action_12887582 ] Daniel Dai commented on PIG-1373: - All the changes are made, need to verify API changes link

[jira] Created: (PIG-1496) Mandatory rule ImplicitSplitInserter

2010-07-12 Thread Daniel Dai (JIRA)
Mandatory rule ImplicitSplitInserter Key: PIG-1496 URL: https://issues.apache.org/jira/browse/PIG-1496 Project: Pig Issue Type: Sub-task Components: impl Affects Versions: 0.8.0

[jira] Created: (PIG-1497) Mandatory rule PartitionFilterOptimizer

2010-07-12 Thread Daniel Dai (JIRA)
Mandatory rule PartitionFilterOptimizer --- Key: PIG-1497 URL: https://issues.apache.org/jira/browse/PIG-1497 Project: Pig Issue Type: Sub-task Components: impl Affects Versions: 0.8.0

[jira] Commented: (PIG-1495) Add -q command line option to set queue name for Pig jobs from command line

2010-07-12 Thread Russell Jurney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887585#action_12887585 ] Russell Jurney commented on PIG-1495: - This doesn't work yet. Doh! Add -q command line

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Yan Zhou
I see. There looks like some disconnect about Scenario 1. To me, all filtering logics that can be pushed above JOIN can be figured out without use of CNF, which is scenario 1; while CNF helps to derive the filtering logic after (or, in your example, below) JOIN, which is Scenario 2. In your

[jira] Commented: (PIG-1478) Add progress notification listener to PigRunner API

2010-07-12 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12887602#action_12887602 ] Richard Ding commented on PIG-1478: --- bq. I don't understand the difference between

Re: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Swati Jain
Hi Yan Thanks for your prompt reply. I did not understand your statement “C1 and C2, or their equivalent, above JOIN can be easily figured out without resorting to CNF”. Consider a LOFilter above a LOJoin. The predicate of LOFilter: ( (c1 10) AND (a3+b3 10) ) OR (c2 == 5) The schema for

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-12 Thread Yan Zhou
In the original expression, let (a3+b3 10) to be true, then it transformed to (c1 10) OR (c2 == 5) ) since TRUE OR anything is still TRUE; TRUE and anything is that anything. You can write a visitor to easily do this type of partial evaluation. (a3+b310) is chosen because it can not be