Re: "One to many" provenance

2017-11-20 Thread Phil H
> Matt > >> On Mon, Nov 20, 2017 at 4:32 PM, Phil H <gippyp...@gmail.com> wrote: >> Hi team, >> >> If I have a processor that takes one input flow file, and then generates >> many flow file outputs as a result (say, one output per line from a >> mul

"One to many" provenance

2017-11-20 Thread Phil H
Hi team, If I have a processor that takes one input flow file, and then generates many flow file outputs as a result (say, one output per line from a multi-line input file), how do I indicate the provenance of the new flow files? I would like to see where they have come from for

Re: "One to many" provenance

2017-11-20 Thread Phil H
processors like >> UnpackContent and QueryRecord that explicitly call >> ProvenanceReporter.fork(). >> >> Regards, >> Matt >> >>> On Mon, Nov 20, 2017 at 4:32 PM, Phil H <gippyp...@gmail.com> wrote: >>> Hi team, >>> >

Re: Classloader issues

2017-11-06 Thread Phil H
Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant) > On 7 Nov 2017, at 4:17 am, Phil H <gippyp...@gmail.com> wrote: > > I added the StandardValidators reference back in and the original error > reoccurs. The offending co

Re: Classloader issues

2017-11-06 Thread Phil H
e, you'll want > to add a debugger at runtime and examine the context classloader as it > goes through to see where it goes wonky. > > What version are you on? > > Thanks > > On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gippyp...@gmail.com> wrote: >> Hi guys, &

Re: Classloader issues

2017-11-06 Thread Phil H
9 am, Bryan Bende <bbe...@gmail.com> wrote: > > It is most likely an issue with the Maven configuration in one of your > modules. > > Can you share your project, or the pom files for the processors, NAR, > and bundle? > > Thanks, > > Bryan > > &g

Re: Classloader issues

2017-11-06 Thread Phil H
NIFI_HOME/lib/ | grep jar > > Want to verify that nifi-utils JAR is actually in your NAR and also > that no other unexpected JARs are in your lib directory. > > Thanks, > > Bryan > > On Mon, Nov 6, 2017 at 12:48 PM, Phil H <gippyp...@gmail.com> wrote: >&g

Re: Classloader issues

2017-11-06 Thread Phil H
; Thanks > > On Mon, Nov 6, 2017 at 1:20 PM, Phil H <gippyp...@gmail.com> wrote: >> Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file >> rather than the NAR file >> >> >> >>> On 7 Nov 2017, at 5:17 am, Phil H <g

Re: Classloader issues

2017-11-06 Thread Phil H
first person to ever make that mistake. > > Nope, not true :) No worries and thanks for following up. I believe > once we have the extension registry in play this sort of thing will be > much easier/simpler. > > Thanks > >> On Mon, Nov 6, 2017 at 1:20 PM, Phil H <gippyp

Re: Classloader issues

2017-11-06 Thread Phil H
Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather than the NAR file > On 7 Nov 2017, at 5:17 am, Phil H <gippyp...@gmail.com> wrote: > > Thanks Bryan, > > This new NAR does not appear in the extensions directory (my oth

Classloader issues

2017-11-06 Thread Phil H
Hi guys, I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has

Re: SSL_ERROR_NO_CYPHER_OVERLAP

2018-05-04 Thread Phil H
Server-76] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (EMAILADDRESS=phil@blah, CN=Phil H, OU=Blah) GET https://nifi1.blah/nifi-api/flow/current-user (source ip: 192.168.10.192) 2018-05-05 14:10:29,310 INFO [NiFi Web Server-76] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for EMAILADDRESS

Re: SSL_ERROR_NO_CYPHER_OVERLAP

2018-05-04 Thread Phil H
ave the "Insufficient permissions" problem, which there is loads of documentation as to how to move forwards. Thanks! Phil On Sat, May 5, 2018 at 8:27 AM, Phil H <gippyp...@gmail.com> wrote: > Further info > > In the httpd installation, I need the private key for the certi

Re: SSL_ERROR_NO_CYPHER_OVERLAP

2018-05-04 Thread Phil H
sed PKCS12 for the keystore was some Jetty debug output telling me that was preferred to JKS. Interestingly it gave me contrary advice for the trust store. On Sat, 5 May 2018 at 08:22, Phil H <gippyp...@gmail.com> wrote: > Hi Andy, > > Sorry - poor use of words (it was late when I wrote the emai

Re: SSL_ERROR_NO_CYPHER_OVERLAP

2018-05-04 Thread Phil H
e...@apache.org > *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On May 4, 2018, at 4:03 AM, Phil H <gippyp...@gmail.com> wrote: > > Hi all, > > I am trying to secure my NiFi inst

Data flow rate exceeding provenance recording rate

2018-05-14 Thread Phil H
Hi gang, I have started receiving this error after perhaps 24 hours of run time. The first queue in our flow has a very large backlog by the time this error arrives. What is odd is that the incoming message rate is fairly constant at all times and while I am watching NiFi during the day, we never

Re: Data flow rate exceeding provenance recording rate

2018-05-14 Thread Phil H
roperty from "org.apache.nifi.provenance.PersistentProvenanceRepository" > to "org.apache.nifi.provenance.WriteAheadProvenanceRepository". > The new implementation provides better throughput, will avoid the > problematic pauses that you're encountering now, > and also is quite a bit faster

Re: Data flow rate exceeding provenance recording rate

2018-05-14 Thread Phil H
better (UI is responsive, and no errors or warnings) but the main GetTCP only seems to be spitting out flowfiles about 1/3rd as fast as it used to (network factors have been eliminated). Any more ideas? On Tue, 15 May 2018 at 09:36, Phil H <gippyp...@gmail.com> wrote: > Thanks Mark, > >

SSL_ERROR_NO_CYPHER_OVERLAP

2018-05-04 Thread Phil H
Hi all, I am trying to secure my NiFi installation. I have a client certificate (nifi1.crt) and the CA for the intranet (ca.crt). I created the trust and keystores as below: keytool -import -trustcacerts -alias nifi1 -file nifi1.crt -keystore server_keystore.p12 -storetype PKCS12 keytool

Re: SSL_ERROR_NO_CYPHER_OVERLAP

2018-05-05 Thread Phil H
usual mistake is to fix the > authorizers.xml file but not delete users.xml and authorizations.xml files > before restarting. Could it be the case here? > > Pierre > > 2018-05-05 6:36 GMT+02:00 Phil H <gippyp...@gmail.com>: > > > No worries Andy, > > > > I

SIGSEGV in Jetty

2018-04-25 Thread Phil H
Hi there, I am getting regular (maybe every ten minutes?) crashes in NiFi 1.3.0. This just started happening unrelated to any change to the software environment (i.e.: we haven't installed new code/processors). It roughly coincides with an increase in flow file throughout and my subsequent use

Re: GC overhead limit exceeded (1.3.0)

2018-07-03 Thread Phil H
gt; > then you can start things until you find the problem. > > thanks > > On Tue, Jul 3, 2018, 8:31 PM Phil H wrote: > > > Hi guys, > > > > I got a backlog, noticed degraded performance, restarted NiFi, and now > the > > web UI won't start. I not

GC overhead limit exceeded (1.3.0)

2018-07-03 Thread Phil H
Hi guys, I got a backlog, noticed degraded performance, restarted NiFi, and now the web UI won't start. I note that I am getting lots of out of memory errors in the log. Is there a way I can clear the backlog from outside the application (deleting files, etc)? TIA, Phil

Re: Programmatically adding processors

2018-01-29 Thread Phil H
te in the UI and view the REST calls made. > > Hope that helps. > > Robert > >> On Mon, Jan 29, 2018, 21:31 Phil H <gippyp...@gmail.com> wrote: >> >> Hi there, >> >> I need to add new properties/routes to RouteOnAttribute and subsequent >> PutFil

Programmatically adding processors

2018-01-29 Thread Phil H
Hi there, I need to add new properties/routes to RouteOnAttribute and subsequent PutFile processors based on an external input (a basic web interface). Hoping someone can point me in the right direction in terms of either config files to change, or API calls to make to simulate what the NiFi

Status history graphs over 24hrs (v1.3.0)

2018-08-02 Thread Phil H
Hi there, Is there a way to plot graphs longer than 24 hours? We are interested in seeing things that happen over the course of a week or so Thanks, Phil

Ideal hardware for NiFi

2018-09-09 Thread Phil H
Hi all, I've been asked to spec some hardware for a NiFi installation. Does anyone have any advice? My gut feel is lots of processor cores and RAM, with less emphasis on storage (small fast disks). Are there any limitations on how many cores the JRE/NiFi can actually make use of, or any other

Re: Ideal hardware for NiFi

2018-09-10 Thread Phil H
nna > > On Mon, Sep 10, 2018 at 4:09 AM Phil H wrote: > > > Hi all, > > > > I've been asked to spec some hardware for a NiFi installation. Does > anyone > > have any advice? My gut feel is lots of processor cores and RAM, with > less > > emphasis on stor

Re: Load issues

2018-10-23 Thread Phil H
Das wrote: > > Did you increase the no. of Concurrent Tasks and Run Duration on slow > processors ? > > Thanks, > Milan Das > > On 10/23/18, 8:26 PM, "Phil H" wrote: > >Hi team, > >My NiFi is struggling to process data (lots of back pres

Load issues

2018-10-23 Thread Phil H
Hi team, My NiFi is struggling to process data (lots of back pressure in various queues) but it is only using a very small amount (2-5%) of CPU according to top. Any ideas?

Re: Load issues

2018-10-23 Thread Phil H
memory, disks), number of nodes in the cluster, JVM heap size, > configuration of repositories, specific processors that are bottle necks, > etc. > > On Tue, Oct 23, 2018 at 10:02 PM Phil H wrote: > > > Yep - although it now has less throughput with 8 threads than it > > p

Zookeeper - help!

2018-09-30 Thread Phil H
Hi guys, Pulling my hair out trying to solve my Zookeeper problems. I have two 1.6.0 servers that I am trying to cluster. Here is the except from the properties files – all other properties are default so omitted for clarity. The servers are set up to run HTTPS, and the interface works via

RE: Zookeeper - help!

2018-10-02 Thread Phil H
.port=9551 - nifi.cluster.node.address=nifi2.com - nifi.cluster.node.protocol.port=11441 nifi2/state/zookeeper /myid (file contents = "2") /state-management.xml (no changes required) /version-2/ Nathan On 10/2/18, 2:07 AM, "Phil H" wrote: Hi Andy, Thanks for the addition

RE: Zookeeper - help!

2018-10-03 Thread Phil H
kind but it looks like it's not: http://zookeeper.apache.org/doc/r3.4.3/zookeeperStarted.html#sc_RunningReplicatedZooKeeper On 10/2/18, 5:17 PM, "Phil H" wrote: The second port in the zookeeper server config has been a mystery to me. I thought it was a second port used for

RE: Zookeeper - help!

2018-10-02 Thread Phil H
://pierrevillard.com/2016/08/13/apache-nifi-1-0-0-cluster-setup/ [2]  https://pierrevillard.com/2016/11/29/apache-nifi-1-1-0-secured-cluster-setup/ Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69 On Oct 1, 2018, at 2:45 PM, Phil H

RE: Zookeeper - help!

2018-10-01 Thread Phil H
uses Zookeeper and how to configure it here: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#state_management. Let us know what happens once these properties are configured correctly. Nathan On 9/30/18, 11:07 PM, "Phil H" wrote: Hi guys, Pulling m

Re: Ideal hardware for NiFi

2018-09-13 Thread Phil H
te to the content repo is 400 MB/sec). Similar with > Provenance Repository. > > Doing this also will allow you to hold a larger 'archive' of content and > provenance data, because > it will span the archive across all of the listed directories, as well. > > Thanks > -Mark > >

RE: Ideal hardware for NiFi

2018-09-13 Thread Phil H
. If iostat shows always 100% usage then prob things arent ideal. thanks On Thu, Sep 13, 2018, 9:16 PM Phil H wrote: > Hi joe, > > I moved the content and providence repositories off to two new disks, but > it seems like the vast majority of the writes are still occurring on the

Re: Ideal hardware for NiFi

2018-09-13 Thread Phil H
ore involved. > > does that help? > > thanks > > On Thu, Sep 13, 2018, 4:36 PM Phil H wrote: > > > Follow up question - how do I transition to this new structure? Should I > > shut down NiFi and move the contents of the legacy single directories > into > >

Re: Ideal hardware for NiFi

2018-09-13 Thread Phil H
, 14 Sep 2018 at 12:37, Joe Witt wrote: > if they are physically seperate the diff should be quite noticable. > > On Thu, Sep 13, 2018, 7:36 PM Phil H wrote: > > > Potentially. We're looking to see how the multiple disks help before > > committing to spending money on new ha

Re: Ideal hardware for NiFi

2018-09-11 Thread Phil H
disk > caching, which > can certainly speed things up, especially if you're reading the content > several times for > each FlowFile. > > Does this help at all? > > Thanks > -Mark > > > > On Sep 10, 2018, at 6:05 AM, Phil H wrote: > > > > Thanks for tha

Re: Ideal hardware for NiFi

2018-09-11 Thread Phil H
gt; writing 100 MB/sec, then > your effective write rate to the content repo is 400 MB/sec). Similar with > Provenance Repository. > > Doing this also will allow you to hold a larger 'archive' of content and > provenance data, because > it will span the archive across all of the listed

Re: Cluster question - interval vs external routing

2019-11-07 Thread Phil H
ps://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration > > > > Best regards, > Endre > > > > On 7 Nov 2019, at 03:49, Phil H wrote: > > > > Hi there, > > > > We have two addresses on each of our NiFi cluster

Cluster question - interval vs external routing

2019-11-06 Thread Phil H
Hi there, We have two addresses on each of our NiFi cluster hosts. One is externally facing, which we use to access the UI, and connect to receiving processors from the wider network. We are trying to configure NiFi to use the internal network for its cluster traffic. However if I use the

Re: Cluster issue

2019-12-04 Thread Phil H
you sure that the certificates have valid SubjectAlternativeName > entries for the proper hostnames? > > Andy LoPresto > alopre...@apache.org > alopresto.apa...@gmail.com > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > On Dec 3, 2019, at 7:53 PM

Cluster issue

2019-12-02 Thread Phil H
Hi there, I (almost) have a secure nifi 1.9.2 cluster of two servers. Server A starts up fine, but Server B fails to start with this error (I cannot copy/paste as it is an offline system) Could not start listening for incoming connections in order to load balance data across the cluster. Please

Singleton data inside processors

2019-12-08 Thread Phil H
Hi there, I have a custom processor that I want to contain some shared data across threads (this data is based on analysis of flow files, so constantly changing). I can’t use a static variable though, because I use the same processor more than once within my flow in different contexts. If there

Re: Singleton data inside processors

2019-12-09 Thread Phil H
quite common in many processors. > > Thanks > > On Sun, Dec 8, 2019 at 7:30 PM Phil H wrote: > > > Hi there, > > > > I have a custom processor that I want to contain some shared data across > > threads (this data is based on analysis of flow files, so constantly

Re: Singleton data inside processors

2019-12-09 Thread Phil H
" > then this is what controller services are designed to accommodate but it is > really important you're careful here as shared state like that is where > bottlenecks can occur. Most controller services are meant for a write > once/read many type pattern. > > Thanks > &g

Forcing cluster traffic to a single node

2019-12-05 Thread Phil H
Hi there, I have a GetTCP processor running on both members of my cluster. I want to force data received by both to a processor running on a single node. At the moment, if I set that processor to run on only the primary, the queue from the GetTCP processor on the secondary node just fills up.

Re: Forcing cluster traffic to a single node

2019-12-05 Thread Phil H
> processor on all nodes, not primary node, because it may not be the primary > node that receives the data. > > Thanks > -Mark > > Sent from my iPhone > > > On Dec 5, 2019, at 9:04 PM, Phil H wrote: > > > > Hi there, > > > > I hav

Re: Cloned processor not loading

2019-12-15 Thread Phil H
s.Processor > > -Mark > > > On Sun, Dec 15, 2019 at 6:09 PM Phil H wrote: > > > Hi there, > > > > I have a working processor, Tom, which I cloned to form a second > processor, > > Jerry. All occurrences of Tom in the code/XML were replaced with Jerry in >

Cloned processor not loading

2019-12-15 Thread Phil H
Hi there, I have a working processor, Tom, which I cloned to form a second processor, Jerry. All occurrences of Tom in the code/XML were replaced with Jerry in the clone. The build process works for Jerry fine, as do the unit tests. However when I load the Jerry into the nifi extensions

Re: Maximum load

2020-09-18 Thread Phil H
s. > > Actually, I usually recommend changing the pool size only when you see that > > all the threads available are used. > > Overall we'd need more details about your flow and observations. > > > > Thanks, > > Pierre > > > > Le ven. 18 s

Maximum load

2020-09-17 Thread Phil H
Hi there, Regardless of what I set in the maximum thread count settings, I never see more than 128 active threads. The CPU load of the system is only 5-8% when NiFi is running at this rate. It’s a Solaris system with 64 cores. Any help getting more performance would be appreciated! Regards,

Re: Maximum load

2020-09-20 Thread Phil H
Various numbers - max of 96 so far. On Sat, 19 Sep 2020 at 21:44, Pierre Villard wrote: > How many concurrent tasks did you set on the two custom processors? > > > > Le sam. 19 sept. 2020 à 02:16, Phil H a écrit : > > > > > Basically I have two custom processors

Re: Maximum load

2020-09-20 Thread Phil H
and see what that shows. > > > > Sent from my iPhone > > > > > On Sep 20, 2020, at 8:30 PM, Phil H wrote: > > > > > > Various numbers - max of 96 so far. > > > > > >> On Sat, 19 Sep 2020 at 21:44, Pierre Villard < > pierre.vi

PutTCP delays when multithreaded

2020-08-13 Thread Phil H
If I have more than one thread running for a PutTCP processor, it seems to add nearly 1 second of latency, even when I am only sending a single flow file through my system. This multiplexing does increase the total throughout of my system (items per hour) but this latency is hard to explain. Any

Policy to clear queues

2020-06-03 Thread Phil H
Hi there, I am trying to stratify my userbase. I need to allow certain users/groups the ability to clear queues, but cannot find the right policy to allow that without also allowing them to delete queues, which I absolutely don’t want to do. Am currently using 1.9.2 (putting off the upgrade

Re: Policy to clear queues

2020-06-04 Thread Phil H
roxied > > from the node being used to access the UI out to the remaining nodes. > > > > -Mark > > > > > > On Thu, Jun 4, 2020 at 6:52 AM Phil H wrote: > > > > > Hi Andy, > > > > > > Thanks for your reply. I don’t recall seein

Re: Policy to clear queues

2020-06-04 Thread Phil H
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > On Jun 4, 2020, at 4:45 PM, Phil H wrote: > > > > The action is available in the menu. I get the following pop up: > > > > *Insufficient Permissions* > > > > *Node nifiX.domain.com:443 &

Re: Policy to clear queues

2020-06-04 Thread Phil H
5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > On Jun 4, 2020, at 4:45 PM, Phil H wrote: > > > > The action is available in the menu. I get the following pop up: > > > > *Insufficient Permissions* > > > > *Node nifiX.domain.com:443 <http://nifiX.doma

Re: Policy to clear queues

2020-06-04 Thread Phil H
70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > On Jun 4, 2020, at 4:22 PM, Phil H wrote: > > > > Hi guys, > > > > So I checked this morning, and the users are members of a group that has > > “modify the data” permission at the root level (and i

Re: Policy to clear queues

2020-06-04 Thread Phil H
; PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > On Jun 3, 2020, at 4:08 PM, Phil H wrote: > > > > Hi there, > > > > I am trying to stratify my userbase. I need to allow certain users/groups > > the ability to clear queues, but cannot fin

SSLPeerUnverifiedException following upgrade from 1.6.0 to 1.13.2

2021-06-10 Thread Phil H
Hi there, I upgraded an older dev setup today from 1.6.0 to 1.13.2. After a couple of config tweaks, it’s “working”, but if I try and access the interface at https://nifi2.domain.blah/ I get a message on screen stating that nifi1.domain.blah is not verified. The logs contain this same message,

Re: SSLPeerUnverifiedException following upgrade from 1.6.0 to 1.13.2

2021-06-11 Thread Phil H
t; verification: > > https://square.github.io/okhttp/changelog_3x/#version-3100 > > Regards, > David Handermann > > On Thu, Jun 10, 2021 at 11:16 PM Phil H wrote: > > > Hi there, > > > > I upgraded an older dev setup today from 1.6.0 to 1.13.2. After a

Re: PutFile to mounted directory

2021-05-13 Thread Phil H
a NFS share, the only way is to mount the share on the nifi nodes. > > Regards, > > Eduardo Fontes > > Em qui., 13 de mai. de 2021 01:11, Phil H escreveu: > > > Hi there, > > > > I want to use PutFile write some flow files out to a network drive o

PutFile to mounted directory

2021-05-12 Thread Phil H
Hi there, I want to use PutFile write some flow files out to a network drive on my Unix system, but fail if the disk is not mounted. I can’t see any way to do this - any ideas? Cheers, Phil

SocketTimeoutException (read timeout) on HTTPS

2021-06-02 Thread Phil H
Hi there, I am getting a lot of these both in the web interface to my servers, and in the cluster communication between the nodes. All other aspects of the servers are fine. TCP connections to NiFi, as well as SSH connections to the servers are stable (running for days at a time). I’m lucky if I

Re: SocketTimeoutException (read timeout) on HTTPS

2021-06-02 Thread Phil H
gt; Also, is NiFi using its own dedicated zookeeper or is it shared with other > services? How many nodes is the zookeeper? > > > > > On Jun 2, 2021, at 7:54 PM, Phil H wrote: > > > > Hi there, > > > > I am getting a lot of these both in the web interface to

Re: SocketTimeoutException (read timeout) on HTTPS

2021-06-03 Thread Phil H
gt; threads, and relax some of the timeouts. > > nifi.cluster.protocol.heartbeat.interval= > nifi.cluster.node.protocol.threads= > nifi.cluster.node.protocol.max.threads= > nifi.cluster.node.connection.timeout= > nifi.cluster.node.read.timeout= > nifi.cluster.node.max.concurrent.requests= > >

Re: Primary node vs shutdown

2021-09-01 Thread Phil H
you > have NiFi's cluster load balancing setup. Then you'd disconnect the node > and shut it down. > > Thanks > Shawn > > -Original Message- > From: Phil H > Sent: Wednesday, September 1, 2021 4:36 PM > To: dev@nifi.apache.org > Subject: Primary node vs shut

Primary node vs shutdown

2021-09-01 Thread Phil H
Hi there, I am noticing a number of situations where shutting down one node in a cluster is leaving data stranded in the flows on that shut down server. Is there any way to tell NiFi to ship data off to other cluster members before it shuts down? Note I am restarting via the nifi.sh script, not

Re: Primary node vs shutdown

2021-09-01 Thread Phil H
t it down. > > Thanks > Shawn > > -Original Message- > From: Phil H > Sent: Wednesday, September 1, 2021 4:36 PM > To: dev@nifi.apache.org > Subject: Primary node vs shutdown > > Hi there, > > I am noticing a number of situations where shutting down

Re: Primary node vs shutdown

2021-09-01 Thread Phil H
And once reconnected again, no data passes that queue - it all just piles up there (the queue count matching the number of items sent into the cluster). However if I try and list the queue, it claims there are no files in it. Very very confused! On Thu, 2 Sep 2021 at 08:39, Phil H wrote: > O

Re: Primary node vs shutdown

2021-09-01 Thread Phil H
. Phil > On 2 Sep 2021, at 12:02 pm, Joe Witt wrote: > > Phil > > What processor reads from that queue that appears unmoving? > > Thanks > > On Wed, Sep 1, 2021 at 3:51 PM Phil H wrote: > >> And once reconnected again, no data passes that queue - it all jus

Re: Primary node vs shutdown

2021-09-04 Thread Phil H
ep 1, 2021 at 7:56 PM Phil H wrote: > > > Hi Joe, > > > > It’s a custom one, but it is effectively just a routing filter component > > (read the data, send the flow file out on relationship A or B based on > what > > it finds). Nothing exotic in terms of how it int

Re: Using nifi in separated networks

2021-08-02 Thread Phil H
Hi Marc, Thanks for the additional info. Just so you know you’re not the only one, I’ve also had to re-implement a ListenTCP alternative to get around the byte delimeter issue for binary and multiline text data. Phil On Tue, Aug 3, 2021 at 6:59 AM Marc wrote: > > Hi Adam, > > more or less it

Re: Using nifi in separated networks

2021-08-02 Thread Phil H
as to your > thought process and design approach. > > OK, I'll step off a little, because I might be probing too hard here. But I > was legitimately curious about the intention of the proposed processor as > it relates to the mentioned Diode device. > > Thanks, > > Adam

Re: Using nifi in separated networks

2021-08-01 Thread Phil H
That is interesting stuff - out of interest, if it was sent over that UDP diode, how would you know whether or not it got to the other side? I haven’t looked into the site-to-site functionality much yet but I assume it maintains the providence info? On Mon, 2 Aug 2021 at 04:26, Marc wrote: >

Issue with setting nifi.sensitive.props.key

2021-12-21 Thread Phil H
Hi there, I am in the process of trying to upgrade from 13.2 to 15.1. I did not have a sensitive props key set previously. Based on the upgrade guide, I ran nifi.sh set-sensitive-properties-key APassword When I ran nifi, it was complaining about a lack of specified algorithm. I ran up a new

Unit tests for multiple processors

2021-07-21 Thread Phil H
Hi there, I use the built in unit tests in the maven processor archetype for each of my processors, but I would like to set up some integration testing using multiple processors. Is this possible? If so, anyone care to sling me some example code? Eg: say I wanted to run a test connecting

Re: Unit tests for multiple processors

2021-07-25 Thread Phil H
://github.com/TouK/plumber > > > > Tony Koval > > > > On 2021/07/22 03:25:31, Phil H mailto:g...@gmail.com>> wrote: > >> Hi there,> > >> > >> I use the built in unit tests in the maven processor archetype for each of> > &g

SplitContent doesn’t support regex?

2022-03-16 Thread Phil H
Hi, This seems like an odd omission - aside from performance (presumably?) is there a reason why there isn’t a regex option for the byte sequence? I need one but thought I’d ask before I built my own. Thanks Phil

Re: SplitContent doesn’t support regex?

2022-03-16 Thread Phil H
ve a good implementation in mind you should go for it. > Sounds interesting. > > Thanks > > On Wed, Mar 16, 2022 at 3:59 PM Phil H wrote: > > > Hi, > > > > This seems like an odd omission - aside from performance (presumably?) is > > there a reason why

Missing maven dependencies when building nifi

2022-04-02 Thread Phil H
Hi there, Trying to build the nifi git repo for the first time and I'm getting the message below I'm used to working in offline development environments, so I'm a bit lost with this online dependency stuff. I note that https://repository.apache.org/snapshots/org/apache/nifi/ doesn't contain any

Re: Missing maven dependencies when building nifi

2022-04-04 Thread Phil H
2\database\migration\H2DatabaseMigrator.java:[37,49] error: cannot find symbol [ERROR] symbol: class JdbcDataSource [ERROR] location: class H2DatabaseMigrator On Sat, Apr 2, 2022 at 10:20 PM Phil H wrote: > Hi there, > > Trying to build the nifi git repo for the first time and I'm getting the

Re: Missing maven dependencies when building nifi

2022-04-04 Thread Phil H
- build it all, run contrib check, and use all the > cores. > > Thanks > > On Mon, Apr 4, 2022 at 6:38 AM Mike Thomsen > wrote: > > > Did you run "mvn install" from the root of the project before > > attempting to run a full test of the code base? > > >

Re: Missing maven dependencies when building nifi

2022-04-04 Thread Phil H
xls = -- some binary stuff -- On Tue, Apr 5, 2022 at 12:30 AM Phil H wrote: > Thanks Joe, it seems to be about half way through now - much further than > previously. I thought I saw some red text flash past, but nothing > stopped. Does

Re: Missing maven dependencies when building nifi

2022-04-04 Thread Phil H
> Thanks > > On Mon, Apr 4, 2022 at 7:46 AM Phil H wrote: > > > So, about that error that flashed past... > > > > [INFO] > > > > [INFO] BUILD FAILURE > > [INFO] > >

Re: Missing maven dependencies when building nifi

2022-04-04 Thread Phil H
25) Re-running resulted in the same failure a second time (points for consistency!), but I'm not sure what to do about this one. Phil On Mon, Apr 4, 2022 at 11:38 PM Mike Thomsen wrote: > Did you run "mvn install" from the root of the project before > attempting to run a full t

Re: SplitContent doesn’t support regex?

2022-04-04 Thread Phil H
in mind you should go for it. > Sounds interesting. > > Thanks > > On Wed, Mar 16, 2022 at 3:59 PM Phil H wrote: > > > Hi, > > > > This seems like an odd omission - aside from performance (presumably?) is > > there a reason why there isn’t a regex optio

Re: Missing maven dependencies when building nifi

2022-04-05 Thread Phil H
> Too many builds in different levels are leaving things around. > > Thanks > > On Mon, Apr 4, 2022 at 7:46 AM Phil H wrote: > > > So, about that error that flashed past... > > > > [INFO] > > ---

Re: Missing maven dependencies when building nifi

2022-04-06 Thread Phil H
, 2022 at 2:31 PM James Srinivasan < > james.sriniva...@gmail.com> > wrote: > > > Possibly not your problem, but did you follow the instructions for > building > > on Windows here: > > > > https://nifi.apache.org/quickstart.html > > > > On Tue, 5

Re: IT WORKS

2022-04-07 Thread Phil H
Oh, and maven on my laptop works fine now that it’s on my home WiFi and not running via an iOS hotspot. Go figure... > On 7 Apr 2022, at 4:34 pm, Phil H wrote: > > Upgraded to Azul JDK11 on my VM and 1.17.0 now runs! > > I am re-running the full mvn clean install process

Re: Missing maven dependencies when building nifi

2022-04-06 Thread Phil H
Australia. Never had any other issues with OpenSSL stuff, etc On Wed, 6 Apr 2022 at 21:11, Otto Fowler wrote: > What country are you in? Are you under export controls? > > From: Phil H > Reply: dev@nifi.apache.org > Date: April 6, 2022 at 07:08:45 > To: dev@nifi.apac

Re: Missing maven dependencies when building nifi

2022-04-06 Thread Phil H
ption: Illegal key size at org.apache.nifi.security.kms.FileBasedKeyProviderTest.getSecretKeysPath(FileBasedKeyProviderTest.java:60) at org.apache.nifi.security.kms.FileBasedKeyProviderTest.testGetKey(FileBasedKeyProviderTest.java:47) On Wed, Apr 6, 2022 at 5:08 PM Phil H wrote: > Hi guys,

Re: Missing maven dependencies when building nifi

2022-04-06 Thread Phil H
tography Extension Policy. Upgrading to the latest > > Java 8 version should resolve the illegal key size issue shown in the > > build. > > > > Regards, > > David Handermann > > > > On Wed, Apr 6, 2022 at 6:16 AM Phil H wrote: > > > > > Austral

Compiling / Testing a single processor

2022-04-09 Thread Phil H
Hi there, Now that my environment is working nicely, I am working on SplitContent (Nifi-1517). Is there a way to tell maven to only compile and/or run the unit tests for the SplitContent processor, rather than testing the entire nifi-standard-processors bundle? Thanks, Phil

New build problem, Java 17

2022-04-06 Thread Phil H
So I upgraded to the Azul JDK 17. New repo, following the same instructions as before. phil@Phils-MacBook-Pro nifi % mvn -T C2.0 clean install -Pinclude-grpc [INFO] Scanning for projects... Downloading from central:

  1   2   >