[GitHub] [nifi] pvillard31 opened a new pull request #5460: NIFI-9304 - Adding PublishGCPubSubLite and ConsumeGCPubSubLite processors

[GitBox]

pvillard31 opened a new pull request #5460:
URL: https://github.com/apache/nifi/pull/5460


   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
   #### Adding PublishGCPubSubLite and ConsumeGCPubSubLite processors
   
   This pull request is adding two processors to interact with GCP Pub/Sub 
Lite. I initially thought about improving the existing processors but since it 
uses quite different APIs, I decided to go with new processors.
   
   I did some extensive testing in a GCP environment and it seems to be working 
as expected including under high load and with concurrent tasks.
   
   Please note that I had to use the compute engine credentials or the 
application credentials. If providing a service account JSON payload/file in 
the controller service, it seems that some gRPC calls are ignoring this 
authentication method. It seems to be specific to Pub/Sub Lite as I was able to 
use the GCS, PubSub and BigQuery processors using the service account JSON file.
   
   Some screenshots of my tests:
   
   <img width="1417" alt="Screenshot 2021-10-16 at 14 14 57" 
src="https://user-images.githubusercontent.com/11541012/137587930-f45c8304-6aea-444f-8a13-b000169f0fb4.png";>
   
   <img width="1782" alt="Screenshot 2021-10-16 at 14 16 52" 
src="https://user-images.githubusercontent.com/11541012/137587934-a5bf6886-d0b8-4f75-8bc1-68aa2cddd490.png";>
   
   <img width="1785" alt="Screenshot 2021-10-16 at 14 17 06" 
src="https://user-images.githubusercontent.com/11541012/137587939-be564e74-61ff-477e-88c2-1eee2bfb9546.png";>
   
   <img width="756" alt="Screenshot 2021-10-16 at 14 18 03" 
src="https://user-images.githubusercontent.com/11541012/137587947-f7c63425-fe65-496f-adc4-ddd4cebdc533.png";>
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
        in the commit message?
   
   - [ ] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically `main`)?
   
   - [ ] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org