[jira] Commented: (PIG-784) PigStorage() - need ability to turn off Attempt to access field warnings

[Santhosh Srinivasan (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12702524#action_12702524
 ] 

Santhosh Srinivasan commented on PIG-784:
-

By default, these warnings are aggregated and should not appear in the logs. If 
required, the warning aggregation can be turned off.

 PigStorage() - need ability to turn off Attempt to access field  warnings
 ---

 Key: PIG-784
 URL: https://issues.apache.org/jira/browse/PIG-784
 Project: Pig
  Issue Type: Bug
Reporter: David Ciemiewicz

 I want an option to PigStorage() for LOAD which will allow me to turn off the 
 Attempt to access field warnings.
 Something like:
 {code}
 define PigStorage PigStorage(warn_load_nonexistent_field=off);
 A = load 'mydata.txt' using PigStorage()
 as (col1: chararray, col2_optional: int, col3_optional: float);
 {code}
 or
 {code}
 A = load 'mydata.txt' using PigStorage(warn_load_nonexistent_field=0)
 as (col1: chararray, col2_optional: int, col3_optional: float);
 {code}
 If I have a very large data set with optional columns that are not populated 
 (and have no tab separator), I'd like to just read the file as is and not 
 generate the warnings.
 The warnings are problematic because the fill up the logging output and every 
 System.out.println will generate slow down the overall processing.  
 Especially if the data file being processed is missing one or more columns on 
 every single row.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-784) PigStorage() - need ability to turn off Attempt to access field warnings

[David Ciemiewicz (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12702528#action_12702528
 ] 

David Ciemiewicz commented on PIG-784:
--

@Santhosh

Hmmm.  I'm running Pig in local mode with the latest published build and I get 
lots of warnings and they are not aggregated:

-bash-3.00$ pig -exectype local -latest cat.pig
USING: /grid/0/gs/pig/current
2009-04-24 20:02:55,666 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,667 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,672 [main] INFO  
org.apache.pig.backend.local.executionengine.LocalPigLauncher - 100% complete!
2009-04-24 20:02:55,672 [main] INFO  
org.apache.pig.backend.local.executionengine.LocalPigLauncher - Success!!
(a,1,42.0F)
(,,)
(,,)
(,,)
(,,)