Nifi as unprocessed UDP packet forwarder?

2018-10-24 Thread Stephen Greszczyszyn
Hi there, I am planning on using NiFi for various types of data ingestion, but was wondering if I could use NiFi for raw UDP packet forwarding. The data I am trying to route is: 1) Collectd (UDP): I don't think that there is a collectd processor for NiFi, so I'd like to take the raw UDP flows

Recommended NiFi Docker volume mappings?

2018-10-24 Thread Stephen Greszczyszyn
Hi there, I'm trying to get a working configuration for the official vanilla NiFi docker image where it can read existing SFTP incoming data as well as allow me to pass in any necessary configuration files. The problem seems to be that by default the docker container picks up userID 1000 to run

Re: Recommended NiFi Docker volume mappings?

2018-10-25 Thread Stephen Greszczyszyn
g directory. In one day fills some > Gigabytes. If you want to mount logs, adjust the logging. > > On Thu, 25 Oct 2018 at 10:07 Stephen Greszczyszyn > wrote: > >> >> >> On Thu, 25 Oct 2018 at 12:50, Peter Wilcsinszky < >> peterwilcsins...@gmail.com> wr

Re: Recommended NiFi Docker volume mappings?

2018-10-25 Thread Stephen Greszczyszyn
On Thu, 25 Oct 2018 at 12:50, Peter Wilcsinszky wrote: But even with 1.8 I'll need to declare the host mount directory somehow via docker-compose, as how will the built docker image on dockerhub know where to locally mount the internal $(NIFI_HOME) volumes as described below? VOLUME

Re: Recommended NiFi Docker volume mappings?

2018-10-25 Thread Stephen Greszczyszyn
own image from it. > > On Wed, Oct 24, 2018 at 10:04 PM Stephen Greszczyszyn > wrote: > >> Hi there, >> >> I'm trying to get a working configuration for the official vanilla NiFi >> docker image where it can read existing SFTP incoming data as well as al

Re: Debugging NiFi Flows in Docker - problem with ListFile?

2019-05-28 Thread Stephen Greszczyszyn
, Stephen Greszczyszyn wrote: > OK, thanks, this us useful information, I did also test the filenames > with a regex parser, but didn't know NiFi was different. I really don't > need an expression anyway as I want to get all the files. I'm wondering if > the problem is that I have to

Re: Debugging NiFi Flows in Docker - problem with ListFile?

2019-05-28 Thread Stephen Greszczyszyn
> consist of 3 characters, beginning with a c and ending with an r". What you > want > is "the filename must begin with a c, followed by any character, followed > by an r, followed by anything else." So you'd want "c.r.*" I do believe. > > Thanks > -Mar

Re: Debugging NiFi Flows in Docker - problem with ListFile?

2019-05-28 Thread Stephen Greszczyszyn
the entire line of text must match the > regex exactly in order to be selected. So '^c.r' will > not match but '^c.r.*' will. > > Thanks > -Mark > > On May 28, 2019, at 3:15 PM, Stephen Greszczyszyn > wrote: > > I don't think that is the problem as I have checked the regex

Debugging NiFi Flows in Docker - problem with ListFile?

2019-05-28 Thread Stephen Greszczyszyn
Hi there, I'm running NiFi in docker. I'm trying to enable flow debugs and also to determine if NiFi can actually see in my data directory path. I'm trying to use a simple recursive list file processor to find the files based on a regular expression "^c.r". I can docker exec into the container