[
https://issues.apache.org/jira/browse/PIG-3400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13722987#comment-13722987
]
Rohini Palaniswamy commented on PIG-3400:
-----------------------------------------
Ok. Got it. That involves changing the dfs in PigContext and ensuring
everywhere the default filesystem is s3. It would be good to document it if you
do not want to do it. What is the behaviour on EMR pig?
> FS commands do not work with S3 paths
> -------------------------------------
>
> Key: PIG-3400
> URL: https://issues.apache.org/jira/browse/PIG-3400
> Project: Pig
> Issue Type: Bug
> Components: grunt
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: 0.12
>
> Attachments: PIG-3400-2.patch, PIG-3400.patch
>
>
> To reproduce issues, run the following commands w/ S3 paths:
> {code}
> pig -e 'ls s3://<path>'
> pig -e 'mkdir s3://<path>'
> pig -e 'cp s3://<path1> s3://<path2>'
> pig -e 'mv s3://<path1> s3://<path2>'
> pig -e 'copyToLocal s3://<path>/<file> .'
> pig -e 'copyFromLocal <file> s3://<path>'
> {code}
> As of now, none of these commands works in Apache Pig, whereas they do in EMR
> Pig. The problem is that in GruntParser, DataStorage is constructed using the
> default file system provided by configuration, and s3 paths are not
> recognized.
> Instead, we should construct DataStorage based on the given URL. For example,
> {code}
> DataStorage dfs = new HDataStorage(new Path(path).toUri(), mConf);
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
