exceptionfactory opened a new pull request, #11052: URL: https://github.com/apache/nifi/pull/11052
# Summary [NIFI-15754](https://issues.apache.org/jira/browse/NIFI-15754) Adds [Google Cloud Storage](https://cloud.google.com/storage) support to Iceberg modules with a `GCSIcebergFileIOProvider` Controller Service implementation. The new Controller Service is packaged in a separate `nifi-iceberg-gcs` module and bundled in `nifi-iceberg-gcs-nar`, following the pattern of the AWS S3 and Azure Data Lake Storage implementations. Although the [iceberg-gcp](https://central.sonatype.com/artifact/org.apache.iceberg/iceberg-gcp) module from the Apache Iceberg project includes a [GCSFileIO](https://github.com/apache/iceberg/blob/main/gcp/src/main/java/org/apache/iceberg/gcp/gcs/GCSFileIO.java) implementation, the class depends on the [google-cloud-storage](https://central.sonatype.com/artifact/com.google.cloud/google-cloud-storage) library which has dozens of direct and transitive dependencies. Instead of using the `GCSFileIO` class, the Controller Service package includes a direct implementation of the Iceberg `FileIO` interface named `GoogleCloudStorageFileIO`. The implementation uses the Java [HttpClient](https://docs.oracle.com/en/java/javase/21/docs/api/java.net.http/java/net/http/HttpClient.html) for REST operations with the GCS API. Supporting Bearer Token authentication with vended credentials from an Iceberg REST Catalog, the `FileIO` implementation avoids multiple layers of dependencies. The new implementation reads the same properties defined in the Iceberg [GCPProperties](https://github.com/apache/iceberg/blob/main/gcp/src/main/java/org/apache/iceberg/gcp/GCPProperties.java) to support compatibility with Iceberg Catalogs. The `InputStream` and `OutputStream` implementations handle direct interaction with Google Cloud Storage, including support for [resumable uploads](https://docs.cloud.google.com/storage/docs/resumable-uploads). Tests for the `FileIO` implementation include interaction with the [OkHttp MockWebServer](https://square.github.io/okhttp/#mockwebserver) to verify expected HTTP requests and responses. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000` - [X] Pull request contains [commits signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits) with a registered key indicating `Verified` status ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `./mvnw clean install -P contrib-check` - [X] JDK 21 - [X] JDK 25 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
