[
https://issues.apache.org/jira/browse/PIG-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated PIG-145:
---------------------------
Status: Open (was: Patch Available)
Your changes broke the semantics of copyToLocal in grunt. Previously, if a pig
job stored the results to a file (ie store a to 'myfile';) then myfile is
created as a directory. In grunt, a call to copyToLocal on that directory
would then copy the directory and it's files out of the dfs. Now, that call in
grunt results in:
grunt> copyToLocal /user/pig/tests/results/gates.1206814491/Checkin_1.out bla
2008-03-29 12:17:15,115 [main] ERROR org.apache.pig.tools.grunt.GruntParser -
java.lang.ClassCastException: org.apache.pig.backend.hadoop.datastorage.HFile
at
org.apache.pig.backend.hadoop.datastorage.HDirectory.copy(HDirectory.java:80)
at org.apache.pig.backend.hadoop.datastorage.HPath.copy(HPath.java:111)
at
org.apache.pig.tools.grunt.GruntParser.processCopyToLocal(GruntParser.java:421)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:155)
at
org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:73)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:54) at
org.apache.pig.Main.main(Main.java:253)
> LocalFile ignores active container and HDataStorage can't copy to other
> DataStrorage
> ------------------------------------------------------------------------------------
>
> Key: PIG-145
> URL: https://issues.apache.org/jira/browse/PIG-145
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Charlie Groves
> Attachments: PIG-145-DataStorage_Bugs.patch
>
>
> As part of starting to rewrite the DataStorage APIs, I wrote some unit tests
> for the existing DataStorage implementations to make sure I wasn't breaking
> anything. In testing the open code, I found that LocalFile doesn't respect
> the active container you set you LocalDataStorage, so if you open a relative
> file, it's relative to wherever you're running the code. Similarly, while
> testing the copy operations, I found that HFile doesn't allow copying to
> anything other than other HFiles and that HDirectory's copy operation was
> never used because it had the wrong signature.
> The attached patch fixes these issues and adds tests for much of the
> DataStorage API for both of the existing backends. There are no tests for
> the sopen code as I'm planning on changing that significantly in rewriting
> these.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
