[
https://issues.apache.org/jira/browse/PIG-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607330#comment-13607330
]
Feng Peng commented on PIG-3222:
--------------------------------
We can see two different signatures in the output:
{noformat}
2013-03-20 06:08:55,634 [main] INFO org.apache.pig.Main - Apache Pig version
0.11.0+68 (rexported) compiled Nov 09 2012, 22:17:38
2013-03-20 06:08:55,960 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to
hadoop file system at:
2013-03-20 06:08:56,142 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to
map-reduce job tracker at:
setStoreFuncUDFContextSignature is called with signature: samples_1-3
setStoreFuncUDFContextSignature is called with signature: samples_1-6
2013-03-20 06:08:57,276 [main] INFO org.apache.pig.tools.pigstats.ScriptState
- Pig features used in the script: UNKNOWN
2013-03-20 06:08:57,660 [main] INFO hive.metastore - Trying to connect to
HiveMetaStore with URI
2013-03-20 06:08:57,681 [main] INFO hive.metastore - Connected to
HiveMetaStore with URI
2013-03-20 06:08:57,917 [main] ERROR org.apache.hadoop.hive.conf.HiveConf -
Found both hive.metastore.uris and javax.jdo.option.ConnectionURL Recommended
to have exactly one of those config keyin configuration
2013-03-20 06:08:58,045 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File
concatenation threshold: 100 optimistic? false
2013-03-20 06:08:58,069 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size before optimization: 1
2013-03-20 06:08:58,069 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 1
2013-03-20 06:08:58,298 [main] INFO org.apache.pig.tools.pigstats.ScriptState
- Pig script settings are added to the job
2013-03-20 06:08:58,302 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2013-03-20 06:08:58,305 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Using reducer estimator:
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
2013-03-20 06:08:58,954 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
- BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=447650450912
2013-03-20 06:08:58,954 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting Parallelism to 448
2013-03-20 06:09:01,374 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- creating jar file Job8844815194973821141.jar
2013-03-20 06:09:05,589 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- jar file Job8844815194973821141.jar created
2013-03-20 06:09:05,605 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
2013-03-20 06:09:05,611 [main] INFO org.apache.pig.data.SchemaTupleFrontend -
Key [pig.schematuple] is false, will not generate code.
2013-03-20 06:09:05,611 [main] INFO org.apache.pig.data.SchemaTupleFrontend -
Starting process to move generated code to distributed cacche
2013-03-20 06:09:05,611 [main] INFO org.apache.pig.data.SchemaTupleFrontend -
Setting key [pig.schematuple.classes] with classes to deserialize []
2013-03-20 06:09:05,756 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.
setStoreFuncUDFContextSignature is called with signature: samples_1-6
...
{noformat}
> New UDFContextSignature assignments in Pig 0.11 breaks HCatalog.HCatStorer
> ---------------------------------------------------------------------------
>
> Key: PIG-3222
> URL: https://issues.apache.org/jira/browse/PIG-3222
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.11
> Reporter: Feng Peng
> Labels: hcatalog
> Attachments: PigStorerDemo.java
>
>
> Pig 0.11 assigns different UDFContextSignature for different invocations of
> the same load/store statement. This change breaks the HCatStorer which
> assumes all front-end and back-end invocations of the same store statement
> has the same UDFContextSignature so that it can read the previously stored
> information correctly.
> The related HCatalog code is in
> https://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-0.5/hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatStorer.java
> (the setStoreLocation() function).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
