[ 
https://issues.apache.org/jira/browse/PIG-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702528#action_12702528
 ] 

David Ciemiewicz commented on PIG-784:
--------------------------------------

@Santhosh

Hmmm.  I'm running Pig in local mode with the latest published build and I get 
lots of warnings and they are not aggregated:

-bash-3.00$ pig -exectype local -latest cat.pig
USING: /grid/0/gs/pig/current
2009-04-24 20:02:55,666 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,667 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,668 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,669 [main] WARN  
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigHadoopLogger - 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject:
 Attempt to access field which was not found in the input
2009-04-24 20:02:55,672 [main] INFO  
org.apache.pig.backend.local.executionengine.LocalPigLauncher - 100% complete!
2009-04-24 20:02:55,672 [main] INFO  
org.apache.pig.backend.local.executionengine.LocalPigLauncher - Success!!
(a,1,42.0F)
(,,)
(,,)
(,,)
(,,)
(,,)
(,,)
(,,)
(,,)

> PigStorage() - need ability to turn off "Attempt to access field"  warnings
> ---------------------------------------------------------------------------
>
>                 Key: PIG-784
>                 URL: https://issues.apache.org/jira/browse/PIG-784
>             Project: Pig
>          Issue Type: Bug
>            Reporter: David Ciemiewicz
>
> I want an option to PigStorage() for LOAD which will allow me to turn off the 
> "Attempt to access field" warnings.
> Something like:
> {code}
> define PigStorage PigStorage("warn_load_nonexistent_field=off");
> A = load 'mydata.txt' using PigStorage()
>         as (col1: chararray, col2_optional: int, col3_optional: float);
> {code}
> or
> {code}
> A = load 'mydata.txt' using PigStorage("warn_load_nonexistent_field=0")
>         as (col1: chararray, col2_optional: int, col3_optional: float);
> {code}
> If I have a very large data set with optional columns that are not populated 
> (and have no tab separator), I'd like to just read the file as is and not 
> generate the warnings.
> The warnings are problematic because the fill up the logging output and every 
> System.out.println will generate slow down the overall processing.  
> Especially if the data file being processed is missing one or more columns on 
> every single row.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to