Dear Wiki user, You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.
The following page has been changed by TomWhite: http://wiki.apache.org/lucene-hadoop/AmazonS3 The comment on the change is: Update following fixes in 0.10.1 ------------------------------------------------------------------------------ or as a convenient repository for data input to and output from Map/Reduce. In the second case HDFS is still used for the Map/Reduce phase. - S3 support was introduced in Hadoop 0.10 ([http://issues.apache.org/jira/browse/HADOOP-574 HADOOP-574]), + S3 support was introduced in Hadoop 0.10.0 ([http://issues.apache.org/jira/browse/HADOOP-574 HADOOP-574]), - but it needs the patch in [http://issues.apache.org/jira/browse/HADOOP-857 HADOOP-857] to work properly. + but this had a few bugs so you should use Hadoop 0.10.1 or later. The patch in [https://issues.apache.org/jira/browse/HADOOP-862 HADOOP-862] makes S3 work with the Hadoop CopyFile tool. - (Hopefully these patches will be integrated in the next release.) = Setting up hadoop to use S3 as a replacement for HDFS = @@ -75, +74 @@ the `bin/hadoop distcp` tool. Then once the Map/Reduce job is complete the output is copied to S3 as input to a further job, or retrieved as a final result. - [More instruction will be added after [https://issues.apache.org/jira/browse/HADOOP-862 HADOOP-862] is complete.] + [More instructions will be added after [https://issues.apache.org/jira/browse/HADOOP-862 HADOOP-862] is complete.]
