[
https://issues.apache.org/jira/browse/PIG-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Le Dem updated PIG-2699:
-------------------------------
Attachment: PIG-2699_b.patch
PIG-2699_b.patch
with Apache headers
> Pig creates too many instances of Load and Store Funcs. It should be 1 in the
> front-end and 1 in the backend
> ------------------------------------------------------------------------------------------------------------
>
> Key: PIG-2699
> URL: https://issues.apache.org/jira/browse/PIG-2699
> Project: Pig
> Issue Type: Bug
> Components: internal-udfs
> Affects Versions: 0.10.0
> Reporter: Julien Le Dem
> Assignee: Julien Le Dem
> Attachments: PIG-2699.patch, PIG-2699_a.patch, PIG-2699_b.patch
>
>
> Attached: a patch to get it down to 3
> Here is the report of the remaining calls.
> some methods are unnecessarily called multiple times, this should be improved
> as well.
> {noformat}
> A = LOAD 'foo' USING TestLoadStoreFuncLifeCycle$Loader();
> STORE A INTO 'bar' USING TestLoadStoreFuncLifeCycle$Storer();
> 3 instances of Loader
> 3 instances of Storer
> all calls:
> Loader(1).<init>
> Loader(1).relativeToAbsolutePath
> Storer(1).<init>
> Storer(1).setStoreFuncUDFContextSignature
> Storer(1).relToAbsPathForStoreLocation
> Storer(1).setStoreFuncUDFContextSignature
> Loader(1).getSchema
> Loader(1).getSchema
> Loader(1).getSchema
> Loader(1).getSchema
> Loader(1).getSchema
> Loader(1).getSchema
> Storer(1).setStoreFuncUDFContextSignature
> Loader(1).getSchema
> Loader(1).getSchema
> Loader(1).getSchema
> Storer(1).setStoreLocation
> Storer(1).getOutputFormat
> Loader(1).getSchema
> Loader(1).getStatistics
> Loader(1).setLocation
> Storer(1).setStoreLocation
> Storer(2).<init>
> Storer(2).setStoreFuncUDFContextSignature
> Storer(2).setStoreLocation
> Storer(2).getOutputFormat
> Loader(2).<init>
> Loader(2).setUDFContextSignature
> Loader(2).setLocation
> Loader(2).getInputFormat
> Storer(3).<init>
> Storer(3).setStoreFuncUDFContextSignature
> Storer(3).setStoreLocation
> Storer(3).getOutputFormat
> Storer(3).setStoreLocation
> Loader(3).<init>
> Loader(3).setUDFContextSignature
> Loader(3).setLocation
> Loader(3).getInputFormat
> Storer(3).setStoreLocation
> Storer(3).getOutputFormat
> Storer(3).prepareToWrite
> Loader(3).setUDFContextSignature
> Loader(3).prepareToRead
> Loader(3).getNext
> Storer(3).putNext
> Loader(3).getNext
> Storer(3).putNext
> Loader(3).getNext
> Storer(3).putNext
> Loader(3).getNext
> Storer(3).setStoreLocation
> Storer(3).setStoreLocation
> Storer(1).setStoreLocation
> constructor calls:
> Loader(1).<init> called by
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:565)
> org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:421)
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3170)
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1293)
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:791)
> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:509)
> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:384)
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:175)
> org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1602)
> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1549)
> org.apache.pig.PigServer.registerQuery(PigServer.java:534)
> org.apache.pig.PigServer.registerQuery(PigServer.java:547)
> Storer(1).<init> called by
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:565)
> org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:477)
> org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:6336)
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1337)
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:791)
> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:509)
> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:384)
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:175)
> org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1602)
> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1549)
> org.apache.pig.PigServer.registerQuery(PigServer.java:534)
> org.apache.pig.PigServer.registerQuery(PigServer.java:547)
> Storer(2).<init> called by
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:565)
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getStoreFunc(POStore.java:232)
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setLocation(PigOutputFormat.java:168)
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecsHelper(PigOutputFormat.java:200)
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:187)
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:258)
> Loader(2).<init> called by
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:565)
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:254)
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:258)
> Storer(3).<init> called by
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:565)
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.getStoreFunc(POStore.java:232)
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.getCommitters(PigOutputCommitter.java:84)
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.<init>(PigOutputCommitter.java:66)
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:279)
> Loader(3).<init> called by
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:565)
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getLoadFunc(PigInputFormat.java:158)
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:106)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
