Hi Maxime, It seems to be a similar issue as NIFI-8662 [1] where the AWS client library cannot parse the Region from a VPC-style endpoint and we need to get it from the Region property. I will look into it in more detail and get back to you.
Regards, Peter Turcsanyi [1] https://issues.apache.org/jira/browse/NIFI-8662 On Tue, Mar 8, 2022 at 4:05 PM LEZIER Maxime (ITNOVEM) <[email protected]> wrote: > Hello, > > > > I would like to ask you again about this subject, would you have an answer > for me please? > > > > *ML* > > > > > > *De : *LEZIER Maxime (ITNOVEM) <[email protected]> > *Date : *vendredi, 4 mars 2022 à 08:47 > *À : *[email protected] <[email protected]> > *Objet : *[AWS][1.15.3] - Unable to use vpc endpoint > > Hello, > > > > I use nifi in 1.15.3 version. > > > > I have to reach aws ressources (S3,SQS..) which have VPCE in front. > > > > My test pipeline for this usecase is a simple ListS3 with AwsCredentials > controller. > > > > In the listS3 i have set the bucket’s name, the region, point it to the > credentials controller and set an endpoint override url with a url like > this : > > https://bucketname.vpce-xxx-xxx.s3.eu-west-3.vpce.amazonaws.com > > > > For the controller, i set up access id/secret id, the role’s arn i have to > assume, the role assume session name, and the sts vpce endpoint url which > have this form : > > vpce-xxx-xxx.sts.eu-west-3.vpce.amazonaws.com (tried with and without > https:// in front)d > > > > When i start this flow i’ve got this error : > > > > 2022-03-04 08:42:08,067 ERROR [Timer-Driven Process Thread-16] > org.apache.nifi.processors.aws.s3.ListS3 > ListS3[id=501cd602-017f-1000-ffff-ffff9f28cb25] Failed to list contents of > bucket due to > com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: > Credential should be scoped to a valid region, not 'vpce'. (Service: > AWSSecurityTokenService; Status Code: 403; Error Code: > SignatureDoesNotMatch; Request ID: d57a4d34-7e59-41ba-9e71-4505dd801e75; > Proxy: null): > com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: > *Credential should be scoped to a valid region, not 'vpce'*. (Service: > AWSSecurityTokenService; Status Code: 403; Error Code: > SignatureDoesNotMatch; Request ID: d57a4d34-7e59-41ba-9e71-4505dd801e75; > Proxy: null) > > com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: > Credential should be scoped to a valid region, not 'vpce'. (Service: > AWSSecurityTokenService; Status Code: 403; Error Code: > SignatureDoesNotMatch; Request ID: d57a4d34-7e59-41ba-9e71-4505dd801e75; > Proxy: null) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1819) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1403) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1372) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) > > at > com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) > > at > com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) > > at > com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1728) > > at > com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1695) > > at > com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1684) > > at > com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:488) > > at > com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:457) > > at > com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.newSession(STSAssumeRoleSessionCredentialsProvider.java:343) > > at > com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.access$000(STSAssumeRoleSessionCredentialsProvider.java:41) > > at > com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider$1.call(STSAssumeRoleSessionCredentialsProvider.java:90) > > at > com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider$1.call(STSAssumeRoleSessionCredentialsProvider.java:87) > > at > com.amazonaws.auth.RefreshableTask.refreshValue(RefreshableTask.java:257) > > at > com.amazonaws.auth.RefreshableTask.blockingRefresh(RefreshableTask.java:213) > > at > com.amazonaws.auth.RefreshableTask.getValue(RefreshableTask.java:154) > > at > com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.getCredentials(STSAssumeRoleSessionCredentialsProvider.java:315) > > at > com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider.getCredentials(STSAssumeRoleSessionCredentialsProvider.java:40) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1257) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:833) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:783) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704) > > at > com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686) > > at > com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) > > at > com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) > > at > com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5445) > > at > com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5392) > > at > com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5386) > > at > com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:927) > > at > org.apache.nifi.processors.aws.s3.ListS3$S3ObjectBucketLister.listVersions(ListS3.java:544) > > > > > > > > I see than there is some ticket which refer to this issue, and that it > should be resolved : > > https://issues.apache.org/jira/browse/NIFI-5456 > > https://issues.apache.org/jira/browse/NIFI-8662 > > > > I don’t knwon if it’s a misconfiguration from me or if it’s still this bug > concern. > > > > Could you help me to resolve this please ? > > > > Really thanks. > > > > ML > > > > > > Interne > ------- > Ce message et toutes les pièces jointes sont établis à l'intention > exclusive de ses destinataires et sont confidentiels. L'intégrité de ce > message n'étant pas assurée sur Internet, la SNCF ne peut être tenue > responsable des altérations qui pourraient se produire sur son contenu. > Toute publication, utilisation, reproduction, ou diffusion, même partielle, > non autorisée préalablement par la SNCF, est strictement interdite. Si vous > n'êtes pas le destinataire de ce message, merci d'en avertir immédiatement > l'expéditeur et de le détruire. > ------- > This message and any attachments are intended solely for the addressees > and are confidential. SNCF may not be held responsible for their contents > whose accuracy and completeness cannot be guaranteed over the Internet. > Unauthorized use, disclosure, distribution, copying, or any part thereof is > strictly prohibited. If you are not the intended recipient of this message, > please notify the sender immediately and delete it. >
