RE: EXT :Re: Custom S3 endpoint

2018-12-20 Thread Martin, Nick
: Custom S3 endpoint Hi Nick, What version of Hadoop are you using? AFAIK, you must use Hadoop 2.7+ to support custom s3 endpoint, or the `fs.s3a.endpoint` property in core-site.xml would be ignored. Best, Paul Lam 在 2018年12月19日,06:40,Martin, Nick mailto:nick.mar...@ngc.com>> 写道: I’m w

Re: Custom S3 endpoint

2018-12-18 Thread Paul Lam
Hi Nick, What version of Hadoop are you using? AFAIK, you must use Hadoop 2.7+ to support custom s3 endpoint, or the `fs.s3a.endpoint` property in core-site.xml would be ignored. Best, Paul Lam > 在 2018年12月19日,06:40,Martin, Nick 写道: > > I’m working on Flink 1.7.0 and I’m tryi

Custom S3 endpoint

2018-12-18 Thread Martin, Nick
I'm working on Flink 1.7.0 and I'm trying to use the built in S3 libraries like readFile('s3://bucket/object') or StreamingFileSink. My storage provider is not AWS, but they implement the same API. So I need to point the S3 client to a different address. The Hadoop documentation shows that