I see it!Thanks,I want to know how Impala uses S3, I did't see the code in Impala about choosing s3 and invoke s3's API
在 2016年12月14日星期三 UTC+8下午11:47:33，Tim Armstrong写道： > > See https://wiki.apache.org/hadoop/AmazonS3 and S3AFilesystem.java in the > Hadoop source tree. > > It looks like Alluxio also has a similar filesystem implementation. I > don't know anything about Alluxio or how well it works in practce, just > giving pointers in case you are interested in developing the support. > > On Tue, Dec 13, 2016 at 10:01 PM, 黄志 <[email protected] <javascript:>> > wrote: > >> Hi, I am interested in this, I saw Impala use boto3 to implement >> S3Client, but Alluxio only offer Java API,how do we use it?and where is >> S3AFileSystem? >> >> 在 2016年12月9日星期五 UTC+8上午1:34:42，Tim Armstrong写道： >>> >>> It looks like Alluxio provides some kind of Hadoop Filesystem API >>> wrapper: >>> http://www.alluxio.org/docs/master/en/File-System-API.html#hadoop-api. >>> I don't know if it's been upstreamed into Apache Hadoop. >>> >>> We do use similar Hadoop Filesystem API adapters to provide S3 and >>> Isilon support without massive code changes so it seems likely that it's >>> possible to do the same for Alluxio. Our experience with S3 and Isilon was >>> that it required additional work to get it to work and perform well because >>> different filesystems have different quirks even if they implement the same >>> interface. >>> >>> It would require at least some small code changes to switch it on (e.g. >>> you can grep for S3AFilesystem to see how we turned on S3 support). Please >>> let us know if you experiment with it and have any success (or no success). >>> >>> - Tim >>> >>> On Thu, Dec 8, 2016 at 9:07 AM, Jim Apple <[email protected]> wrote: >>> >>>> I don't know that Impala has been tested with Alluxio. >>>> >>>> This mailing list is deprecated; please use >>>> [email protected] >>>> >>>> >>>> On Wednesday, December 7, 2016 at 11:08:59 PM UTC-8, >>>> [email protected] wrote: >>>>> >>>>> Hi >>>>> How are you configure the impala on the alluxio; >>>>> >>>>> Alluxio: Open Source Memory Speed Virtual Distributed Storage >>>>> Alluxio: http://alluxio.org/ >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Impala Dev" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Impala Dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> > >
