Attributes To JSON question

Hi I am using the attributesToJSON processor to create a JSON flowfile which I then send to Elastic, I have noticed that some of my attributes which ar numbers such as file size always come out of the attributesToJSON processor as string values (ie with double quotes around them), therefore

Merging the unique attributes of 2 flowfiles

Hi I have a group of log files coming in via http listener, up to 30 logs  per transaction, of which I only need the values that are in 2 of those log files per transaction. After using some RouteOnContents I end up with the two log flowfiles I want. In my current flow I am using a

Counting the number of occurrences of a phrase

Good morning devs I have a flowfile which contains a space separated log file, within this file is a section such as:- FileSelectors:[{fileselector:},{fileselector:},{fileselector:}] I would like to create an attribute which is the count the number of occurrences of the word fileselector and

Re: Record Readers and Writers

-scripting-nar/1.11.4/org.apache.nifi.record.script.ScriptedReader/index.html [6] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-scripting-nar/1.11.4/org.apache.nifi.record.script.ScriptedRecordSetWriter/index.html Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com

Record Readers and Writers

Hi I want to use the ConvertRecord Processor with it's underlying Record Readers and Writers to convert files from XML or JSON to a bespoke format and probably vice versa.I have looked at the Readers/Writers currently provided and decided that I can use the XML/JSON ones provided but I will

Re: Advanced UI button on Update Attributes

le, too? nifi-update-attribute-ui/src/main/webapp/META-INF/nifi-processor-configuration Thanks, Koji On Thu, Jan 3, 2019 at 7:29 PM DAVID SMITH wrote: > > Hi > I am looking to create a bespoke processor based very loosely on the Update Attribute processor set, I have take

Advanced UI button on Update Attributes

Hi I am looking to create a bespoke processor based very loosely on the Update Attribute processor set, I have taken a copy of the update attribute bundle and changed the name of the bundle and its components to nifi-test-build-processor, nifi-test-build-model etc, I have also amended the pom

Re: Help with loading a file into a cache

, "WHI8765", "WHT8765" ] }} ZA103:{"Aircraft Type":"Wessex", "Lifed Items":{ "port engine":"RR-P7654", "starboard engine":"RR-S7654","gearboxes":[ "WHM7654", "WHI7654", &qu

Help with loading a file into a cache

Hi Devs I am running a NiFi 1.8 cluster, each node has 128Gb of Ram. I need to load the contents of a file of which is around 5Gb in size  into a Key/Value cache. The file I want to load is produced by another company so the format it comes in is not negotiable. The file contains thousands of

Re: Accumulo processors

er. He works on HBase and Accumulo and mentioned to me that he might be up for a code review on the Accumulo processors. On Sun, May 27, 2018 at 4:38 PM DAVID SMITH <davidrsm...@btinternet.com.invalid> wrote: > Mike > Thanks for the suggestion I will certainly have a look at the HBase code

Re: Accumulo processors

won't help you with the Accumulo APIs, but it could provide useful strategies for how to identify and assign labels from user input. On Sun, May 27, 2018 at 3:14 PM DAVID SMITH <davidrsm...@btinternet.com.invalid> wrote: > Mark > Thanks for the link, I have downloaded th

Re: Accumulo processors

Mark Thanks for the link, I have downloaded the code from Github, it will be a good basis to start with. Many thanksDave On Saturday, 26 May 2018, 20:22, Mark Payne wrote: Hi Dave, I do have a branch in Github with the work that I had done:

Help & advice needed on application.js in Processors

Hi I am trying to create a processor that is partially based on the UpdateAttribute processor 0.7.3.I have cloned the UpdateAttribute source, renamed the processor and I have started by trying to amend the Advanced configuration UI. I have found that I can change labels for fields in the

Re: Setting up an SSL context in Minifi 0.3.0

mplate, config.yml, and logs as possible for your instance exhibiting problems.  Please be mindful to strip out any sensitive information. [1] https://issues.apache.org/jira/browse/MINIFI-408 [2] https://issues.apache.org/jira/browse/MINIFI-403 Thanks, Aldrin On Thu, Jan 18, 2018 at 10:56 AM, DAVID

Setting up an SSL context in Minifi 0.3.0

Hi I put up a post a little while ago, because I was having trouble getting an SSL context working in Minifi 0.2.0Aldrin advised to wait until 0.3.0 came out as some dev work had been done in this area. I have trying it again today and still cannot get an SSL context to work in a ListenHTTP

Re: Syslog processing from cisco switches to Splunk

rocessor, not the PutSplunk processor? > > On Thu, Oct 19, 2017 at 7:31 AM, DAVID SMITH <davidrsm...@btinternet.com> > wrote: > >> Hi >> We are trying to do something which on the face of it seems fairly simple >> but will not work.We have a cisco switch which i

Syslog processing from cisco switches to Splunk

Hi We are trying to do something which on the face of it seems fairly simple but will not work.We have a cisco switch which is producing syslogs, normally we use zoneranger to send them to Splunk and the records are shown.However we want to do a bit of content routing, so we are using NiFi

Help with Flowfile Content to Attribute

Hi I have a scenario where I have a flowfile which is come in and I want to email the contents to a recipient. I am currently using the putSmtp processor which works fine except that I have to email the flowfile as an attachment, so when the recipient gets the email they have to open an

Deconstructing SMTP /Mime emails

Hi I have a scenario where I use listenSMTP to listen for emails, and then I use ExtractEmailHeaders and ExtractEmailAttachments to deconstruct these emails into their constituent parts, however, I don't seem to be able to get the body text of the email as either an attribute or a FlowFile. Can

Help with back porting Email bundle from NiFi 1.1.1 to NiFi 0.7.2

Hi At work we are in need of a set of email/smtp processors, our system currently uses NiFi 0.7.2  on Centos 7 and Java 8. We can't upgrade the NiFi version at the present time.I found a set of processors which will do the job we require in NiFi 1.1.1, I have tried running the 1.1.1 nar

Help with ExtractText processor and Mime format files

:26:12 -0400 (EDT) From: David Smith <d...@home.example.com> To: John Doe <j...@example.com> Subject: =?iso-8859-1?Q?Test Email?= Message-ID: <pine.msc.4.21.0001112233.8672-2345...@home.example.com> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-123456789-98

NiFi Elastic Clustering

Hi We are looking to set up some NiFi Clusters, however, we would like them to scale elastically without having to restart the NCM as we add or remove nodes. >From NiFi 0.7.1 onwards is this possible or is this a feature that is being >looked at or should we be looking at our own solution? Many

ExecuteScript and flowfiles

Hi I have a question from a colleague who is using nifi 0.5.1, he has some files coming in which he needs to break up into 3 constituent parts. He has a python script which should do this but what he wants to know is can he either 1)   send all three parts down one relationship as three

Re: Problems with links on the Contributors Guide

matter. For the checkstyle configuration, if I remember correctly there are two links pointing to the configuration file but only one is valid. Did you manage to add the checkstyle conf? Pierre 2016-02-23 16:02 GMT+01:00 DAVID SMITH <davidrsm...@btinternet.com>: > Hi > As I am looking to su

Problems with links on the Contributors Guide

Hi As I am looking to submit code I looked on the Contributors Guide at the Code Style section.I am using Eclipse Mars and I loaded the Checksytyle plugin, I downloaded the eclipse-formatter.xml from the link, but it will not load into the Check Configuration panel in Eclipse, I get the

Extending the NiFi API

Hi At work we run many instances of NiFi and other associated data applications, with more instances of NiFi being planned it is becoming impracticable to use the UI's to add, delete users or amend their roles. We have developed an in house tool which controls these associated applications via

Re: [GitHub] nifi pull request: NIFI-865 added initial support for AMQP publish...

er implementation used, so it must be neutral. > > Thanks for reaching out once again. > Cheers > Oleg > >> On Feb 6, 2016, at 5:19 PM, DAVID SMITH <davidrsm...@btinternet.com> wrote: >> >> Hi Guys >> |    | This email has been sent from a virus-free comp

Re: [GitHub] nifi pull request: NIFI-865 added initial support for AMQP publish...

Hi Guys || This email has been sent from a virus-free computer protected by Avast. www.avast.com | As you may remember I have developed some processors that publish/subscribe to AMQP brokers, but I was having problems writing Junit tests for these processors. I was interested to see that

Testing processor logging message in Junit tests

Hi Does anyone know how to test for logging messages generated by my processors in my Juint tests for that processor?I am trying to determine whilst testing my processor if I have entered an exception, where I would just normally log a warning or an Error but necessarily end up on the failure

Re: XML Questions

Joe Thanks for the reply, that is pretty much what I thought about the flowfile attributes. Ill have to have a think about that one to see if I can work out a way around it. With the merging, we will get multiple xml files which will be related by an attribute, I want to combine them into one

XML Questions

Hi I have two questions which I think may require bespoke processors. 1)  I have extracted a set of  values from xml tags and made them into flowfile attributes, how a can I add these flowfile attributes to one or more flowfiles? 2)  I have multiple xml files which I need to merge into one xml

[DISCUSS] Job Code entry feature for graph changes

Hi  I was doing a demonstration of NiFi to a new audience at work yesterday, the audience were most impressed and could see many uses, however two major questions arose. Audit and Accounting is a major thing where I work, and we would like to track who made changes and why.   I think I have

Re: Advanced UI on a Processor

custom UI is for. Make sure that has your processors type specified. Let me know if that helps. Matt Sent from my iPhone > On Sep 12, 2015, at 3:59 PM, DAVID SMITH <davidrsm...@btinternet.com> wrote: > > Hi > I have to create a processor that dedupes  some values held in

Re: Update Attribute Advanced rules not persisting

, 2015 at 4:20 PM, DAVID SMITH davidrsm...@btinternet.com wrote: Hi I have an instance of nifi-0.2.1 running in Centos 6.6, I have used an Update Attribute processor to set some rules, I can save the rules ad the processor runs as expected. However, if I go back into the processor configuration

Re: Writing to a flowfile

Mark Thanks very much for all of your help, that works really well, I have also taken on board your other comments and implemented them on my home version.  I will use it all at work tomorrow. As you may have seen on a post I made in July, I have taken the put get JMS processors and made a

Re: Instantiating a Controller Service in a Junit test

Mark Thanks for the information, it works a treat. Dave On Monday, 10 August 2015, 1:12, Mark Payne marka...@hotmail.com wrote: David, Yes, you'll also need to set the controller service in your processor. Sorry, I forgot to mention that. So after the call to