Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2553
Thanks for the update.I have merged my changes that came from your code
review into the NIFI-4914-rebase PR as well. That PR has extends the base
Pulsar processor capabilities to include
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2553
At this risk of sounding stupid, can you spell out the steps you are
suggesting? I want to make sure I don't screw up the process. I will close
this PR once I "rebase 4914 and squa
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2553
So the goal is to get my 23 commits squashed down into a single commit?
---
Github user david-streamlio closed the pull request at:
https://github.com/apache/nifi/pull/2553
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2553
Alright, sounds good.
---
Github user david-streamlio closed the pull request at:
https://github.com/apache/nifi/pull/2554
---
GitHub user david-streamlio opened a pull request:
https://github.com/apache/nifi/pull/2614
Added Apache Pulsar Processors and Controller Service
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
One single squashed commit, with both NIFI-4908 & NIFI-4914 changes in it.
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
So, is this a change in the 1.7.x code base or is it already in the 1.6.0
code? I created my fork back on Feb 22nd based on the 1.6.0-SNAPSHOT branch,
which does not have these enums
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
I don't see an upstream branch labeled 1.6.0which upstream branch
should I merge into my local branch?
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
@MikeThomsen I will work on those changes now and have them completed
ASAPThanks for your patience
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
Yea, I'm not sure what happened
First I did the following 4 steps you recommended:
As a rule of thumb, this is how you want to do this sort of update:
git che
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
My latest commit passed all the tests, and is showing no conflicts with the
base branch. But you are seeing "an explosion of conflicts". So I am
confusedWhat steps do you
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
run the rebase against my 'master' branch or my 'NIFI-4914' branch?
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
`Davids-MacBook-Pro:nifi david$ git checkout NIFI-4914
Switched to branch 'NIFI-4914'
Your branch is up to date with 'origin/NIFI-4914'.
Davids-MacBook-Pro:ni
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
`Davids-MacBook-Pro:nifi david$ git rebase
First, rewinding head to replay your work on top of it...
Applying: NIFI-4289 - InfluxDB Put processor
Applying: NIFI-4827 Added support
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
Davids-MacBook-Pro:nifi david$ git checkout NIFI-4914
Switched to branch 'NIFI-4914'
Your branch is up to date with 'origin/NIFI-4914'.
Davids-MacBook-Pro:ni
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
Davids-MacBook-Pro:nifi david$ git rebase --continue
Applying: Changed artifact versions to 1.7.0-SNAPSHOT
Applying: Fixed issues identified during code review
Applying: Removed
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
Well, that didn't go as well as we had hoped.
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
Followed all of the steps as you suggested. Performed a mvn -Pcontrib-check
clean install without errors.
Now we are failing CI due to missing content under the
nifi-nar-bundles
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
I tend to agree with @joewitt on thisLet's start from a fresh master
and add in the new nifi-nar-bundleIt shouldn't be this hard to add in a
completely fresh bundle, as the
GitHub user david-streamlio opened a pull request:
https://github.com/apache/nifi/pull/2702
Added Apache Pulsar processors
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following
Github user david-streamlio closed the pull request at:
https://github.com/apache/nifi/pull/2614
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2614
Abandoning this PR in favor of PR #2702
https://github.com/apache/nifi/pull/2702
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2702
Looks like there are error in the Email processors causing the CI build to
fail.
[ERROR] Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
1.996 s <<< FAIL
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2702
> Also, something to think about is whether Pulsar's client will work well
across different broker versions. For example, when Pulsar 2.x comes out, will
the 1.x client work well aga
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2702
@MikeThomsen I have updated the code to use the new Apache Pulsar 2.0
client API, but am blocked on the Apache release vote before the the client jar
is published to the Maven central repo
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2702
@MikeThomsen
I have updated my code to use the Apache Pulsar 2.0 client API, but cannot
commit the changes yet, as the release vote is still pending, and thus the
client jar file
GitHub user david-streamlio opened a pull request:
https://github.com/apache/nifi/pull/2490
Added Pulsar processors and Controller Service
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure
Github user david-streamlio closed the pull request at:
https://github.com/apache/nifi/pull/2490
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2490
I neglected to add the JIRA ticket to the commit. NiFi-4908
---
GitHub user david-streamlio opened a pull request:
https://github.com/apache/nifi/pull/2493
Added Pulsar processors and Controller Service
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2493
NIFI-4908
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2493
I have updated all the nifi component version references to 1.6.0-SNAPSHOT,
and successfully rebuilt the project using mvn -Pcontrib-check clean install.
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2493
When I started writing these processors, I used the Kafka ones as a model.
However, I felt that it would be easier to configure the Pulsar client once,
such as SSLContextService integration
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2493#discussion_r171295857
--- Diff:
nifi-nar-bundles/nifi-pulsar-client-services/nifi-pulsar-client-service-api/src/main/java/org/apache/nifi/pulsar/PulsarClientPool.java
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2493
Thanks @markap14. This has been a great learning experience for me, and I
am happy to contribute back to such a great project. I am still new to
developing processors, so I really appreciate
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2493
I created a separate JIRA for record-based
processors.https://issues.apache.org/jira/browse/NIFI-4914, but would like
to get these processors into the release as well. Primarily because
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2493
@markap14. I completely agree that is the best approach for handling
text-based payloads into and out of Pulsar, and am starting to work on the
record based processors now. However, I do see
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2493
The Pulsar client API, https://pulsar.apache.org/api/client/, currently
only supports byte[] payloads. By true streaming, I am assuming you mean an API
that return an IO stream object of some
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2493
I completely agree on your observation about having to land the entire,
unbroken file into NiFi's repo rather than streaming through memory is
inefficient. I will talk to the Pulsar
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2493
At the risk of sounding pushy.Is there anything else I need to do on
this PR, such as add the record-based processors?
---
GitHub user david-streamlio opened a pull request:
https://github.com/apache/nifi/pull/2540
Nifi 4914
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2493
Sure, I will start re-basing now.
---
GitHub user david-streamlio opened a pull request:
https://github.com/apache/nifi/pull/2553
Nifi 4908 rebase
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been
Github user david-streamlio closed the pull request at:
https://github.com/apache/nifi/pull/2493
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2493
I have rebased my branch and created new pull request here:
https://github.com/apache/nifi/pull/2553. Please ignore this pull request as
the new one supersedes it.
---
GitHub user david-streamlio opened a pull request:
https://github.com/apache/nifi/pull/2554
Nifi 4914 rebase
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2553
@MikeThomsen Is there another car bundle that uses a Docker image for
integration testing that I can use as an example?
I will clean up the formatting / style errors.
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2553
@MikeThomsen We have a pre-configured Docker image that has Pulsar if you
want to use that for testing. See https://streaml.io/docs/getting-started. for
detailed instructions.
---
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2553#discussion_r174898281
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-client-service/src/main/java/org/apache/nifi/pulsar/StandardPulsarClientPool.java
Github user david-streamlio closed the pull request at:
https://github.com/apache/nifi/pull/2540
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2540
This branch is based on NIFI-4908 which you are also reviewing. Therefore,
once we complete that code review, I will cherry pick those commits into this
branch.
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2702
Just wanted to check in to see if there is anything more I needed to do on
my end, or if the testing instructions were clear enough.
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2702
Does that mean this commit won't make the 1.7 release?
---
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r197495330
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-client-service/src/main/java/org/apache/nifi/pulsar/StandardPulsarClientService.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r197498285
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-client-service/src/main/java/org/apache/nifi/pulsar/StandardPulsarClientService.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r197505730
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/AbstractPulsarConsumerProcessor.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r197508790
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/AbstractPulsarConsumerProcessor.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r197577680
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/AbstractPulsarProducerProcessor.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r197578241
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/AbstractPulsarProducerProcessor.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r197619121
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/AbstractPulsarConsumerProcessor.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r197620107
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/pubsub/ConsumePulsar.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r197625631
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/pubsub/PublishPulsar.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r197844157
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/pubsub/ConsumePulsar.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r197851635
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/AbstractPulsarProducerProcessor.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r198317582
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/AbstractPulsarProducerProcessor.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2702#discussion_r199829042
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/AbstractPulsarProducerProcessor.java
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2702
From the errors I am seeing in the CI log, it appears that this PR is being
built against the 1.8.0-SNAPSHOT release? Is that correct? Should I change the
version in all of my POMs
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2702
I tried rebase-ing against master, but it had no effect. I think that is
because my "master" is a fork of the 1.7.0 branch... Anyways, here is the
output of the rebas
Github user david-streamlio closed the pull request at:
https://github.com/apache/nifi/pull/2702
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2702
I am going to close this PR and create a new one based on a fresh fork of
the project
---
GitHub user david-streamlio opened a pull request:
https://github.com/apache/nifi/pull/2882
Added Apache Pulsar Processors
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2882
Any update on this? I don't want to miss another release window if
possible... =)
---
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2882#discussion_r207643777
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-client-service/src/main/java/org/apache/nifi/pulsar/StandardPulsarClientService.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2882#discussion_r207644839
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-client-service/src/main/java/org/apache/nifi/pulsar/StandardPulsarClientService.java
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2882
Any update on this? Am I safe to assume we are going to make the 1.8
release?
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2882
FWIW, this PR includes the following Record-Oriented processors:
org.apache.nifi.processors.pulsar.pubsub.ConsumePulsarRecord
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2882#discussion_r219709622
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/AbstractPulsarConsumerProcessor.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2882#discussion_r219709639
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/AbstractPulsarConsumerProcessor.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2882#discussion_r219709641
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/pom.xml ---
@@ -0,0 +1,78 @@
+
+
+http://maven.apache.org/POM/4.0.0
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2882
@rumbin I am merging in changes from @pvillard31, and correcting some minor
issues that cause the processors to hang in certain situations. ETA is by
Monday 11/12 for my next commit with
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2882
Thanks @joewitt I was double comparing the nifi-pulsar-bundle/pom.xml to
other bundles in the nifi-nar-bundles, e.g. nifi-redis-bundle/pom.xml,
nifi-rethinkdb-bundle/pom.xml, and nidi-parquet
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2882
@joewitt So the build doesn't work for you locally? I am able to build it
locally.
The latest commit was trying suggestion number 2 from the following wiki
page that is giv
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2882
As a sanity check I did the following on my local MacBook, and was able to
build NiFi successfully from my branch:
cd /tmp
git clone g...@github.com:david-streamlio/nifi.git
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2882
Is there a way to get this into a 1.8.x release rather than the 1.9.x?
---
Github user david-streamlio closed the pull request at:
https://github.com/apache/nifi/pull/2882
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2882
@rumbin I am going to submit a new PR, it is just easier to create a fresh
branch than rebasing to a new version of NiFi.
---
GitHub user david-streamlio opened a pull request:
https://github.com/apache/nifi/pull/3178
NIFI-4914: Add Apache Pulsar processors
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/2882
Created new PR after rebasing to NiFi 1.9.0-Snapshot branch.
https://github.com/apache/nifi/pull/3178
---
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/3178
@joewitt I have incorporated all of the changes / corrections that came
from both your comments as well as @pvillard31's
[changes](https://github.com/apache/nifi/pull/2882#issueco
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/3178
@joewitt I dug into the build logs and found that the issue with the
parallel build was due to having the wrong version for both of the Pulsar NAR
files defined in the NiFi-assembly/pom.xml
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3178#discussion_r235107339
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-client-service-api/src/main/java/org/apache/nifi/pulsar/cache/LRUCache.java
---
@@ -0,0
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3178#discussion_r235107594
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-client-service/src/main/java/org/apache/nifi/pulsar/StandardPulsarClientService.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3178#discussion_r235123261
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-processors/src/main/java/org/apache/nifi/processors/pulsar/AbstractPulsarProducerProcessor.java
Github user david-streamlio commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3178#discussion_r235123524
--- Diff:
nifi-nar-bundles/nifi-pulsar-bundle/nifi-pulsar-client-service/src/main/java/org/apache/nifi/pulsar/StandardPulsarClientService.java
Github user david-streamlio commented on the issue:
https://github.com/apache/nifi/pull/3178
@joewitt What was the command you used to generate the error above? I want
to attempt to reproduce it locally if possible.
The basic issue is that since this test message was sent
97 matches
Mail list logo
