Hi, I cc Kostas who should be able to help you.
Best, Dawid On 16/01/2019 08:51, Vinay Patil wrote: > Hi, > > Can someone please help on this issue. We have even tried to set > fs.s3a.impl in core-site.xml, still its not working. > > Regards, > Vinay Patil > > > On Fri, Jan 11, 2019 at 5:03 PM Taher Koitawala [via Apache Flink User > Mailing List archive.] <ml+s2336050n25464...@n4.nabble.com > <mailto:ml%2bs2336050n25464...@n4.nabble.com>> wrote: > > Hi All, > We have implemented S3 sink in the following way: > > StreamingFileSink sink= StreamingFileSink.forBulkFormat(new > Path("s3a://mybucket/myfolder/output/"), > ParquetAvroWriters.forGenericRecord(schema)) > .withBucketCheckInterval(50l).withBucketAssigner(new > CustomBucketAssigner()).build(); > > The problem we are facing is that StreamingFileSink is > initializing S3AFileSystem class to write to s3 and is not able to > find the s3 credentials to write data, However > other flink application on the same cluster use "s3://" paths are > able to write data to the same s3 bucket and folders, we are only > facing this issue with StreamingFileSink. > > Regards, > Taher Koitawala > GS Lab Pune > +91 8407979163 > > > ------------------------------------------------------------------------ > If you reply to this email, your message will be added to the > discussion below: > > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/StreamingFileSink-cannot-get-AWS-S3-credentials-tp25464.html > > To start a new topic under Apache Flink User Mailing List > archive., email ml+s2336050n1...@n4.nabble.com > <mailto:ml%2bs2336050n1...@n4.nabble.com> > To unsubscribe from Apache Flink User Mailing List archive., click > here > > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx>. > NAML > > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > >
signature.asc
Description: OpenPGP digital signature
