[
https://issues.apache.org/jira/browse/NIFI-15734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shubham Sharma updated NIFI-15734:
----------------------------------
Description:
h3. Problem
After NIFI-14277 was merged, ConsumeGCPubSub, PublishGCPubSub, and BigQuery
processors fail with {{ACCESS_TOKEN_SCOPE_INSUFFICIENT}} error when using GCP
Workload Identity Federation with Service Account Impersonation.
{code:java}
com.google.api.gax.rpc.UnauthenticatedException:
io.grpc.StatusRuntimeException: UNAUTHENTICATED: Failed computing credential
metadata
Caused by: com.google.api.client.http.HttpResponseException: 403 Forbidden
POST
https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/[SA_EMAIL]:generateAccessToken
{
"error": {
"code": 403,
"message": "Request had insufficient authentication scopes.",
"reason": "ACCESS_TOKEN_SCOPE_INSUFFICIENT",
"domain": "googleapis.com",
"metadata": {
"service": "iamcredentials.googleapis.com",
"method": "google.iam.credentials.v1.IAMCredentials.GenerateAccessToken"
}
}
}{code}
h3. Root Cause
NIFI-14277 introduced explicit OAuth scopes for GCP processors to support
private endpoints. The implementation uses narrow scopes:
* {{GOOGLE_CLOUD_PUBSUB_SCOPE}}
({{{}[https://www.googleapis.com/auth/pubsub]{}}}) for PubSub processors
* {{GOOGLE_CLOUD_BIGQUERY_SCOPE}}
({{{}[https://www.googleapis.com/auth/bigquery]{}}}) for BigQuery processors
When using Workload Identity Federation with Service Account Impersonation, the
authentication flow is:
# OIDC token → STS token exchange → Service Account Impersonation → Final
access token
The impersonation step calls the IAM Credentials API
({{{}iamcredentials.googleapis.com/generateAccessToken{}}}), which requires
{{cloud-platform}} scope. The narrow {{pubsub}} or {{bigquery}} scopes are
insufficient to call this API, causing the authentication failure.
h3. Reproduction Steps
# Configure GCP Workload Identity Federation with Service Account Impersonation
# Create an external account credential configuration file pointing to an OIDC
token file
# Configure GCPCredentialsControllerService with the external account
credential file
# Configure ConsumeGCPubSub or PublishGCPubSub processor to use the credential
service
# Start the processor
# Observe the {{ACCESS_TOKEN_SCOPE_INSUFFICIENT}} error when credentials
refresh/impersonation occurs
was:
h3. Problem
After NIFI-14277 was merged, ConsumeGCPubSub, PublishGCPubSub, and BigQuery
processors fail with {{ACCESS_TOKEN_SCOPE_INSUFFICIENT}} error when using GCP
Workload Identity Federation with Service Account Impersonation.
{code:java}
com.google.api.gax.rpc.UnauthenticatedException:
io.grpc.StatusRuntimeException: UNAUTHENTICATED: Failed computing credential
metadata
Caused by: com.google.api.client.http.HttpResponseException: 403 Forbidden
POST
https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/[SA_EMAIL]:generateAccessToken
{
"error": {
"code": 403,
"message": "Request had insufficient authentication scopes.",
"reason": "ACCESS_TOKEN_SCOPE_INSUFFICIENT",
"domain": "googleapis.com",
"metadata": {
"service": "iamcredentials.googleapis.com",
"method": "google.iam.credentials.v1.IAMCredentials.GenerateAccessToken"
}
}
}{code}
h3. Root Cause
NIFI-14277 introduced explicit OAuth scopes for GCP processors to support
private endpoints. The implementation uses narrow scopes:
* {{GOOGLE_CLOUD_PUBSUB_SCOPE}}
({{{}https://www.googleapis.com/auth/pubsub{}}}) for PubSub processors
* {{GOOGLE_CLOUD_BIGQUERY_SCOPE}}
({{{}https://www.googleapis.com/auth/bigquery{}}}) for BigQuery processors
When using Workload Identity Federation with Service Account Impersonation, the
authentication flow is:
# OIDC token → STS token exchange → Service Account Impersonation → Final
access token
The impersonation step calls the IAM Credentials API
({{{}iamcredentials.googleapis.com/generateAccessToken{}}}), which requires
{{cloud-platform}} scope. The narrow {{pubsub}} or {{bigquery}} scopes are
insufficient to call this API, causing the authentication failure.
h3. Reproduction Steps
# Configure GCP Workload Identity Federation with Service Account Impersonation
# Create an external account credential configuration file pointing to an OIDC
token file
# Configure GCPCredentialsControllerService with the external account
credential file
# Configure ConsumeGCPubSub or PublishGCPubSub processor to use the credential
service
# Start the processor
# Observe the {{ACCESS_TOKEN_SCOPE_INSUFFICIENT}} error when credentials
refresh/impersonation occurs
> GCP PubSub/BigQuery processors fail with ACCESS_TOKEN_SCOPE_INSUFFICIENT when
> using Workload Identity Federation with Service Account Impersonation
> ---------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-15734
> URL: https://issues.apache.org/jira/browse/NIFI-15734
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 2.3.0, 2.8.0
> Reporter: Shubham Sharma
> Priority: Major
>
> h3. Problem
> After NIFI-14277 was merged, ConsumeGCPubSub, PublishGCPubSub, and BigQuery
> processors fail with {{ACCESS_TOKEN_SCOPE_INSUFFICIENT}} error when using GCP
> Workload Identity Federation with Service Account Impersonation.
> {code:java}
> com.google.api.gax.rpc.UnauthenticatedException:
> io.grpc.StatusRuntimeException: UNAUTHENTICATED: Failed computing credential
> metadata
> Caused by: com.google.api.client.http.HttpResponseException: 403 Forbidden
> POST
> https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/[SA_EMAIL]:generateAccessToken
> {
> "error": {
> "code": 403,
> "message": "Request had insufficient authentication scopes.",
> "reason": "ACCESS_TOKEN_SCOPE_INSUFFICIENT",
> "domain": "googleapis.com",
> "metadata": {
> "service": "iamcredentials.googleapis.com",
> "method": "google.iam.credentials.v1.IAMCredentials.GenerateAccessToken"
> }
> }
> }{code}
> h3. Root Cause
> NIFI-14277 introduced explicit OAuth scopes for GCP processors to support
> private endpoints. The implementation uses narrow scopes:
> * {{GOOGLE_CLOUD_PUBSUB_SCOPE}}
> ({{{}[https://www.googleapis.com/auth/pubsub]{}}}) for PubSub processors
> * {{GOOGLE_CLOUD_BIGQUERY_SCOPE}}
> ({{{}[https://www.googleapis.com/auth/bigquery]{}}}) for BigQuery processors
> When using Workload Identity Federation with Service Account Impersonation,
> the authentication flow is:
> # OIDC token → STS token exchange → Service Account Impersonation → Final
> access token
> The impersonation step calls the IAM Credentials API
> ({{{}iamcredentials.googleapis.com/generateAccessToken{}}}), which requires
> {{cloud-platform}} scope. The narrow {{pubsub}} or {{bigquery}} scopes are
> insufficient to call this API, causing the authentication failure.
> h3. Reproduction Steps
> # Configure GCP Workload Identity Federation with Service Account
> Impersonation
> # Create an external account credential configuration file pointing to an
> OIDC token file
> # Configure GCPCredentialsControllerService with the external account
> credential file
> # Configure ConsumeGCPubSub or PublishGCPubSub processor to use the
> credential service
> # Start the processor
> # Observe the {{ACCESS_TOKEN_SCOPE_INSUFFICIENT}} error when credentials
> refresh/impersonation occurs
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
