i solved it by myself, with the help of some debugging. i used s3:///mybucket/ but it needs to be s3://mybucket/some_folder ... 2 slashes, and also a folder needs to be specified ... --
*Fabian WollertZalando SE* E-Mail: fabian.woll...@zalando.de Location: ZMAP <http://zmap.zalando.net/?q=fabian.woll...@zalando.de> Am Do., 31. Mai 2018 um 09:31 Uhr schrieb Fabian Wollert <fab...@zalando.de >: > I'm running it in docker on EC2, cant use EMR ... yes i followed those > instructions. > > Cheers > > -- > > > *Fabian WollertZalando SE* > > E-Mail: fabian.woll...@zalando.de > > > Am Do., 31. Mai 2018 um 03:07 Uhr schrieb Bowen Li <bowenl...@gmail.com>: > >> Did you run Flink on AWS EMR or somewhere else? Have you read and >> followed instructions on >> https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/deployment/aws.html#amazon-web-services-aws >> ? >> >> >> >> On Wed, May 30, 2018 at 7:08 AM, Fabian Wollert <fab...@zalando.de> >> wrote: >> >>> Hi, I'm trying to set up Checkpoints for Flink Jobs with S3 as a >>> filesystem backend. I configured the following: >>> >>> state.backend=filesystem >>> state.backend.fs.checkpointdir=s3:///mybucket/ >>> state.checkpoints.dir=s3:///mybucket/ >>> state.checkpoints.num-retained=3 >>> >>> I also copied the flink-s3-fs-hadoop-1.5.0.jar into the lib folder. >>> >>> I get now though the following error message: >>> >>> Caused by: java.lang.NullPointerException: null uri host. >>> at java.util.Objects.requireNonNull(Objects.java:228) >>> at >>> org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.fs.s3native.S3xLoginHelper.buildFSURI(S3xLoginHelper.java:65) >>> at >>> org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:165) >>> at >>> org.apache.flink.fs.s3hadoop.S3FileSystemFactory.create(S3FileSystemFactory.java:133) >>> >>> I tried to dig deeper into the source code, but struggled to find >>> >>> - what is meant with this URI >>> - where to configure it >>> >>> Can anybody give some advice how to set up the S3 Backend with the new >>> shaded lib jar? >>> >>> Thanks in advance >>> -- >>> >>> >>> *Fabian WollertZalando SE* >>> >>> E-Mail: fabian.woll...@zalando.de >>> >>> Tamara-Danz-Straße 1 >>> <https://maps.google.com/?q=Tamara-Danz-Stra%C3%9Fe+1+10243+Berlin&entry=gmail&source=g> >>> 10243 Berlin >>> <https://maps.google.com/?q=Tamara-Danz-Stra%C3%9Fe+1+10243+Berlin&entry=gmail&source=g> >>> Fax: +49 (0)30 2759 46 93 >>> E-mail: legalnot...@zalando.co.uk >>> Notifications of major holdings (Sec. 33, 38, 39 WpHG): +49 (0)30 >>> 2000889349 >>> >>> Management Board: >>> Robert Gentz, David Schneider, Rubin Ritter >>> >>> Chairman of the Supervisory Board: >>> Lothar Lanz >>> >>> Person responsible for providing the contents of Zalando SE acc. to Art. >>> 55 RStV [Interstate Broadcasting Agreement]: Rubin Ritter >>> Registered at the Local Court Charlottenburg Berlin, HRB 158855 B >>> VAT registration number: DE 260543043 >>> >> >>
