Re: RegEx not catching all tags

2016-06-02 Thread Andy LoPresto
Thanks Sven. Could I ask you to open a Jira [1] requesting a boolean option in the ExtractText processor properties that allows for global results? [1] https://issues.apache.org/jira/browse/NIFI Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F

Re: Merge multiple flowfiles

2016-06-02 Thread Andy LoPresto
incoming connection to MergeContent. Please let us know if this helps, and if not, please share a template and some sample input if possible. Thanks. [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.MergeContent/index.html Andy LoPresto alopre

Re: Wildcard character in the Command Argument field of the ExecuteStreamCommand processor

2016-05-31 Thread Andy LoPresto
would not need to set the working directory Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On May 31, 2016, at 7:02 AM, Huagen peng <huagen.p...@gmail.com> wrote: > > Hi, I would like to run a

Re: Need to measure start and end time for a processor

2016-05-27 Thread Andy LoPresto
. This will include the Kafka batching time, as the provenance metrics are NiFi-only. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On May 27, 2016, at 2:43 PM, Lee Laim <lee.l...@gmail.com> wrote:

Re: Combining outputs in parallel returns a random output

2016-06-02 Thread Andy LoPresto
e final result of the additive operation into the destination processor. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jun 1, 2016, at 22:25, Kavinaesh Dheenadayalan <kavina...@gmail.com> wro

Re: Combining outputs in parallel returns a random output

2016-06-02 Thread Andy LoPresto
] first. I am not guaranteeing this will work, but I would encourage you to explore it. [1] http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.MergeContent/index.html [2] https://nifi.apache.org/docs/nifi-docs/html/user-guide.html Andy LoPresto alopre

Re: Wildcard character in the Command Argument field of the ExecuteStreamCommand processor

2016-06-01 Thread Andy LoPresto
gspot.com/2016/02/invokescriptedprocessor-hello-world.html> [4] http://funnifi.blogspot.com/2016/02/writing-reusable-scripted-processors-in.html <http://funnifi.blogspot.com/2016/02/writing-reusable-scripted-processors-in.html> Andy LoPresto alopre...@apache.org alopresto.ap

Re: Wildcard character in the Command Argument field of the ExecuteStreamCommand processor

2016-05-31 Thread Andy LoPresto
/nifi/processors/script/ExecuteScript.java <https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Finge

Re: Nifi Combiner Processor?

2016-06-22 Thread Andy LoPresto
Shaun, Following the MergeContent, you can do a ReplaceText transform to remove the duplicate message headers by matching on a regular expression. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jun 22, 2

Re: IDE-specific setup

2016-06-21 Thread Andy LoPresto
tps://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-CodeStyle> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jun 21, 2016, at 9:57 AM, Kumiko Yada <kumiko.y...@ds-iq.com> wr

Re: Nifi Combiner Processor?

2016-06-22 Thread Andy LoPresto
tps://funnifi.blogspot.com/2016/03/executescript-json-to-json-revisited_14.html> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jun 22, 2016, at 2:02 PM, Shaun McAdams <shaun.mcad...@moserit.com> wrote: > >

Re: nifi 0.4.1 does not start

2016-01-14 Thread Andy LoPresto
Thanks Frank. Glad to hear it is resolved. Good luck using NiFi. Andy LoPresto alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan 14, 2016, at 3:31 PM, Frank Martínez <mnesa...@gmail.com> wrote: > > Hi Andy, > > Tha

Re: NiFi logging

2016-02-10 Thread Andy LoPresto
rocessor/AbstractProcessor.java [4] http://www.slf4j.org/legacy.html Andy LoPresto alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 10, 2016, at 7:54 PM, sudeep mishra <sudeepshekh...@gmail.com> wrote: > > Hi, > > Wha

Re: File Upload to a ListenHTTP Processor

2016-02-05 Thread Andy LoPresto
Andrew, You may also want to change the content type you are providing. I understand you may not know the format a priori, but “form-data” is probably not what you want. Look at `-H "Content-Type: text/xml”` or `-H “Content-Type: application/octet-stream”` for arbitrary binary data.

Re: Nifi api - SSL_ERROR_WEAK_SERVER_EPHEMERAL_DH_KEY

2016-02-02 Thread Andy LoPresto
org/Security/Server_Side_TLS#DHE_and_Java Andy LoPresto alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 2, 2016, at 12:28 PM, Chakrader Dewaragatla > <chakrader.dewaraga...@lifelock.com> wrote: > > Andy – > > http://

Re: Regular Expressions

2016-03-01 Thread Andy LoPresto
I use Patterns on OS X. It hasn’t been updated in a few years but it’s done everything I need it to. https://itunes.apple.com/us/app/patterns-the-regex-app/id429449079?mt=12 Andy LoPresto alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Ma

Re: Can't use PutS3Object with user names containing commas

2016-03-19 Thread Andy LoPresto
tps://issues.apache.org/jira/secure/CreateIssue!default.jspa> Andy LoPresto alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Mar 18, 2016, at 7:55 AM, Joseph E. Gottman <jgott...@proteuseng.com> > wrote: > > I'm trying to use

Re: HTTP and OAuth

2016-04-12 Thread Andy LoPresto
istributedMapCache, then your other flows can fetch the > token for the desired operations? Alternatively, if you are to provide a > callback for the OAuth service, you could point it at a HandleHttpRequest > endpoint for further processing. > > Andy LoPresto (my go-to guru for all things s

Re: Starting Nifi encounters "unexpected error"

2016-05-16 Thread Andy LoPresto
the repositories when recovering from a server failure unless further debugging demonstrates it is absolutely required. NiFi is fairly resilient about resuming processing after the machine is restored. Let us know if this does not help. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP

Re: Where can I find documentation of implementing custom processor?

2016-04-20 Thread Andy LoPresto
t;https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Apr 20, 2016, at 3:42 PM, Keith Lim <keith@ds-iq.com> wrote: > > Is ther

Re: nifi process running at 114% of node CPU !!?

2016-07-14 Thread Andy LoPresto
-docs/html/administration-guide.html#configuration-best-practices Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jul 14, 2016, at 10:36 AM, Mike Harding <mikeyhard...@gmail.com> wrote: > > ps

Re: Passwords in EncryptContent

2016-07-14 Thread Andy LoPresto
Password enc{3832072F21CB7448E3C268287AF7F2F38359EB7F7E0F22CCFF2432EBC358335A0616B5CDCC895827FE0B1478901D364A} Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jul 14, 2016, at 1:23 PM, Hite, Brett <brett

Re: Passwords in EncryptContent

2016-07-22 Thread Andy LoPresto
i/processors/standard/util/crypto/PBKDF2CipherProviderGroovyTest.groovy#L303 <https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/groovy/org/apache/nifi/processors/standard/util/crypto/PBKDF2CipherProviderGroovyTest.groovy#L303> An

Re: Using ReplaceText to insert newline character into flowfile?

2016-08-03 Thread Andy LoPresto
llTheData) outputStream.write(allTheData.getBytes(StandardCharsets.UTF_8)) } as StreamCallback) log.info("Add backslash R with Groovy all done! (before session.transfer) ") session.transfer(flowFile, REL_SUCCESS) Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprin

Re: Nifi startup fails with EncryptionOperationNotPossibleException

2016-07-13 Thread Andy LoPresto
property like a key or password (specifically from your stacktrace, it does appear to be a processor property). Did the flow document change at all? Did you change the “nifi.sensitive.props.key” value in nifi.properties? Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP

Re: Increase the number of threads available to a Node

2016-07-18 Thread Andy LoPresto
Stéphane, I think this is a great idea. Would you please create a Jira for this feature? The threshold for frequency may need to be configurable, and if the alert icon is clicked or hovered, it could offer a quick description of the issue and how to resolve it. Andy LoPresto alopre

Re: Strategy To collect from hundreds of servers?

2016-08-09 Thread Andy LoPresto
additional advice. [1] https://nifi.apache.org/minifi/ <https://nifi.apache.org/minifi/> [2] https://cwiki.apache.org/confluence/display/MINIFI/Design <https://cwiki.apache.org/confluence/display/MINIFI/Design> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprin

Re: GetTwitter - Security/Certificate Issue

2017-01-31 Thread Andy LoPresto
%3D%20%22Apache%20NiFi%22 <https://issues.apache.org/jira/browse/NIFI-1477?jql=text%20~%20%22truststore%22%20AND%20project%20=%20%22Apache%20NiFi%22> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan

Re: Custom processors in MiNiFi

2017-01-27 Thread Andy LoPresto
://scikit-learn.org/stable/ <http://scikit-learn.org/stable/> [2] https://spark.apache.org/mllib/ <https://spark.apache.org/mllib/> [3] https://funnifi.blogspot.com Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF

Re: Custom processors in MiNiFi

2017-01-27 Thread Andy LoPresto
One other note; you may find additional help on our developers list - d...@nifi.apache.org. This list is more focused on user issues and functionality, while that list gets much deeper into the weeds on coding. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC

Re: Past month data insertion is possible

2017-01-26 Thread Andy LoPresto
Prabhu, I answered this question with an ExecuteScript example which will do what you are looking for on Stack Overflow [1]. [1] http://stackoverflow.com/a/41887397/70465 Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B

Re: Deleting an empty group fails

2017-01-27 Thread Andy LoPresto
Makoto, For future reference, you only need to delete the templates contained in the process group you are trying to delete. You can see the process group ID in the templates listing — any that match the UUID of the PG you are deleting need to be deleted, but others are fine. Andy LoPresto

Re: Deleting an empty group fails

2017-01-27 Thread Andy LoPresto
Created NIFI-3417 [1] to address this. [1] https://issues.apache.org/jira/browse/NIFI-3417 Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan 27, 2017, at 7:35 PM, Joe Witt <joe.w...@gmail.com&

Re: ListenTCP to receive CSV stream.

2017-01-25 Thread Andy LoPresto
blogspot.com/2016/02/executescript-processor-replacing-flow.html <https://funnifi.blogspot.com/2016/02/executescript-processor-replacing-flow.html> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan

Re: NiFi 1.1.0 stuck starting, no errors

2017-01-26 Thread Andy LoPresto
006.html#BABEHABG> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan 26, 2017, at 12:42 PM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > > I’m looking for help in troubleshooting my NiFi

Re: Dynamically set the RabbitMQ routing key

2017-01-25 Thread Andy LoPresto
Brian, I don’t see a reason why this field could not support Expression Language. Please file a Jira requesting this feature [1]. [1] https://issues.apache.org/jira/browse/NIFI <https://issues.apache.org/jira/browse/NIFI> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.c

Re: Microsoft Access file

2017-01-24 Thread Andy LoPresto
) and then treat the Avro records as normal. If you cannot, you may want to explore ExecuteScript and use a JDBC driver which supports Access files [1] to extract the data that way. [1] http://ucanaccess.sourceforge.net/site.html <http://ucanaccess.sourceforge.net/site.html> Andy LoPresto

Re: How to find average of two lines in NiFi?

2017-02-21 Thread Andy LoPresto
(uses Java Reflection)). Once you have the desired result in an attribute, use ReplaceText to update the flowfile content, and MergeContent to merge the individual flowfiles back into a single instance. [1] http://stackoverflow.com/a/42381999/70465 Andy LoPresto alopre...@apache.org alo

Re: NiFi Users: Powered by NiFi page

2017-02-21 Thread Andy LoPresto
Adding Hashmap, Inc. from Chris Herrera: > Begin forwarded message: > > From: Chris Herrera <chris.herrer...@gmail.com> > Subject: Re: Powered by NiFi > Date: February 21, 2017 at 12:00:15 PM PST > To: Andy LoPresto <alopre...@apache.org> > > No problem at al

Re: Split csv file into multipart.

2017-02-20 Thread Andy LoPresto
Prabhu, I answered this question and provided a template on StackOverflow [1]. [1] http://stackoverflow.com/a/42353526/70465 <http://stackoverflow.com/a/42353526/70465> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65

Re: Issue with a simple flow

2017-02-17 Thread Andy LoPresto
can get that value from the Split and update it dynamically, as it could obviously be different for each input. Hopefully Koji or someone else on the list will have a better solution. [1] https://github.com/apache/nifi/pull/1420 <https://github.com/apache/nifi/pull/1420> Andy LoPresto

Re: Means to access StandardHttpContextMap for monitoring purposes

2017-02-17 Thread Andy LoPresto
to be a PMC/committer/dev/user/consistent user of NiFi. Any constructive feedback that helps the project improve is always appreciated. [1] https://issues.apache.org/jira/secure/CreateIssue!default.jspa <https://issues.apache.org/jira/secure/CreateIssue!default.jspa> Andy LoPresto alopre...@apac

Re: Deadlocks after upgrade from 0.6.1 to 1.1.1

2017-02-17 Thread Andy LoPresto
<https://issues.apache.org/jira/browse/NIFI-3364> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 17, 2017, at 8:31 AM, Russell Bateman <r...@windofkeltia.com> wrote: > > Mikhail

Re: How to avoid this splitting of single line as multi lines in SplitText?

2017-02-17 Thread Andy LoPresto
Wes, Do you mind raising a Jira [1] and providing a PR with your fix once you have it translated? I know people would appreciate it. Thanks. [1] https://issues.apache.org/jira/secure/CreateIssue!default.jspa <https://issues.apache.org/jira/secure/CreateIssue!default.jspa> Andy LoPresto

Re: No Response from NiFi to Http POST Request

2017-02-17 Thread Andy LoPresto
Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 16, 2017, at 11:27 AM, James McMahon <jsmcmah...@gmail.com> wrote: > > I am unable to get a response from my NiFi server HandleHttpRequest proc

Re: No Response from NiFi to Http POST Request

2017-02-17 Thread Andy LoPresto
There’s a typo in there. The short flag for —insecure in curl is -k, not -i (—include). Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 17, 2017, at 4:52 PM, Andy LoPresto <alopre...@apache.org&

Re: How to avoid this splitting of single line as multi lines in SplitText?

2017-02-16 Thread Andy LoPresto
a working flow once I applied the patch. All of this is updated on the StackOverflow question as well. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 15, 2017, at 2:19 AM, prabhu Mahendran <prab

Re: Nifi taking forever to start

2017-02-15 Thread Andy LoPresto
If this is not the issue, can you try starting NiFi and then run the following command to generate a thread dump and provide that to the lists? It will greatly help us determine the issue you are encountering. Thanks. $ jcmd Thread.print or $ ./bin/nifi.sh dump filewithdump.txt Andy LoPresto

Re: Exporting and importing workflows, and restoring from backup snapshots

2017-02-15 Thread Andy LoPresto
Jim, If you have or create an account with Confluence, you can receive notifications. There is a “Watch” button at the top right of each page which will subscribe you to email notifications when the page changes. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint

Re: NiFi Users: Powered by NiFi page

2017-02-14 Thread Andy LoPresto
Thank you Jeremy and Hammad. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 14, 2017, at 2:37 PM, Hammad <raoham...@gmail.com> wrote: > > Hi Joe, > Here you go. > > Name: Flexil

Re: NIFI

2017-02-14 Thread Andy LoPresto
client and invoking it via Groovy in an ExecuteScript processor. 3. If that is not sufficient, you would have to build a custom processor and include Thrift as a dependency. Someone else may have more experience, but I would try in that order (from easiest to hardest). Good luck. Andy LoPresto

Re: How to avoid this splitting of single line as multi lines in SplitText?

2017-02-14 Thread Andy LoPresto
Prabhu, I answered this on Stack Overflow [1] but I think you could do it with ReplaceText before the SplitText using a regex like "(.*?)(\n)(.*?)" replaced with "$1\\n$3" [1] http://stackoverflow.com/a/42242665/70465 <http://stackoverflow.com/a/42242665/70465

Re: Nifi taking forever to start

2017-02-15 Thread Andy LoPresto
1/wlcp/wlss30/configwlss/jvmrand.html <https://docs.oracle.com/cd/E13209_01/wlcp/wlss30/configwlss/jvmrand.html> [3] http://www.2uo.de/myths-about-urandom/ <http://www.2uo.de/myths-about-urandom/> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B

Re: NiFi Users: Powered by NiFi page

2017-02-16 Thread Andy LoPresto
Documentation that this was ok: > Begin forwarded message: > > From: dan young <danoyo...@gmail.com> > Subject: Re: Apache NiFi + Looker > Date: February 16, 2017 at 11:59:16 AM PST > To: Andy LoPresto <alopre...@apache.org> > > Sure, i think that's fine. &

Re: Add Attributes to a Flow File in ExecuteScript via Python Stream Callback?

2017-01-16 Thread Andy LoPresto
(flowfile, REL_SUCCESS) Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan 16, 2017, at 2:56 PM, James McMahon <jsmcmah...@gmail.com> wrote: > > In the example you offer Andy, it seems like

Re: Add Attributes to a Flow File in ExecuteScript via Python Stream Callback?

2017-01-16 Thread Andy LoPresto
ntent with ${stringInterpolation.toString()} and math ${ 3 * 2}”) flowFile = session.putAttribute(flowFile, ’message.REMARKS’, “Here I am replacing the prior attribute value with this static string. ") session.transfer(flowFile, REL_SUCCESS) Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC

Re: Add Attributes to a Flow File in ExecuteScript via Python Stream Callback?

2017-01-16 Thread Andy LoPresto
ing-bundle/nifi-scripting-processors/src/test/resources/jython/test_update_attribute.py#L63> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan 16, 2017, at 2:07 PM, Andy LoPresto <alopre...@apach

Re: Splitting Array out of JSON

2017-01-16 Thread Andy LoPresto
ovided input as you enter the expression. Andy LoPresto alopre...@apache.org <mailto:alopre...@apache.org> alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan 16, 2017, at 4:38 PM, Sven Davison &

Re: Splitting Array out of JSON

2017-01-16 Thread Andy LoPresto
Do you have the return type in the processor properties set as Scalar or JSON? You need to select JSON in order to get the results you are looking for. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan

Re: send http request to nifi input port

2017-02-28 Thread Andy LoPresto
Mohammed, NiFi Input Ports are definitely not the answer you are looking for here. As stated, they are specifically for NiFi to NiFi communication using Site to Site protocol, either over raw sockets or HTTPS. They do not accept arbitrary HTTP requests from other entities. Andy LoPresto

Re: send http request to nifi input port

2017-02-27 Thread Andy LoPresto
the file contents. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 27, 2017, at 2:08 PM, mohammed shambakey <shambak...@gmail.com> wrote: > > This is an example of the output file from &qu

Re: send http request to nifi input port

2017-02-27 Thread Andy LoPresto
to be ingested by NiFi, HandleHTTPRequest or (less likely) ListenHTTP are the processors you would want to use. If you can clarify your use case, we should be able to offer more assistance. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E

Re: send http request to nifi input port

2017-02-27 Thread Andy LoPresto
Input Port components as a destination for an arbitrary HTTP request, this will not work. The input ports are designed to work with NiFi’s Site to Site protocol (either over raw sockets or HTTP(S)). Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F

Re: read in values from nifi.properties in a Groovy ExecuteScript processor

2016-09-23 Thread Andy LoPresto
/src/main/java/org/apache/nifi/properties/NiFiPropertiesLoader.java> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Sep 23, 2016, at 3:28 PM, Tom Gullo <tomgu...@gmail.com> wrote: > > I'm us

Re: read in values from nifi.properties in a Groovy ExecuteScript processor

2016-09-23 Thread Andy LoPresto
roovy/org/apache/nifi/properties/NiFiPropertiesLoaderGroovyTest.groovy> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Sep 23, 2016, at 7:45 PM, Andy LoPresto <alopre...@apache.org> wrote: > > To

Re: read in values from nifi.properties in a Groovy ExecuteScript processor

2016-09-24 Thread Andy LoPresto
I do. Those unit tests help a lot. Thanks > > -Tom > >> On Fri, Sep 23, 2016 at 10:47 PM, Andy LoPresto <alopre...@apache.org> wrote: >> Meant to add that there are Groovy unit tests for those classes so you may >> be able to copy the code directly from [1] and

Re: Routeonattribute

2016-10-07 Thread Andy LoPresto
, you’ll want to use *literal* to evaluate the expression. See the Expression Language Guide [1] for more details. [1] https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#literal Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6

Re: ELT on Nifi

2016-10-07 Thread Andy LoPresto
ng-reusable-scripted-processors-in.html <https://funnifi.blogspot.com/2016/02/writing-reusable-scripted-processors-in.html> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 7, 2016, at 7:20 AM, João H

Re: Routeonattribute

2016-10-07 Thread Andy LoPresto
and routed on the relationship name, and that worked as well. Good luck. [1] https://gist.github.com/alopresto/e0458804c642339c5bb32dd9a008f2c5 <https://gist.github.com/alopresto/e0458804c642339c5bb32dd9a008f2c5> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint

Re: SelectHiveQL Error

2016-10-07 Thread Andy LoPresto
? Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 7, 2016, at 6:26 PM, Nathamuni, Ramanujam <rnatham...@tiaa.org> wrote: > > Andrew, > > I agree but reality of enterprise to move one versi

Re: Configure Multiple NCM

2016-09-18 Thread Andy LoPresto
Hi Tijo, Take a look at Mark Payne's article here: http://hortonworks.com/blog/apache-nifi-1-0-0-zero-master-clustering/ Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Sep 18, 2016, at 09:14, Tijo Tho

Re: Requesting Obscene FlowFile Batch Sizes

2016-09-20 Thread Andy LoPresto
Bryan, That’s a good point. Would running with a larger Java heap and higher swap threshold allow Peter to get larger batches out? Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Sep 20, 2016, at 1:41

Re: dynamic getTwitter ?

2016-08-25 Thread Andy LoPresto
/docs/nifi-docs/components/org.apache.nifi.processors.twitter.GetTwitter/index.html [6] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.script.ExecuteScript/index.html [7] https://issues.apache.org/jira/secure/CreateIssue!default.jspa Andy LoPresto alopre...@apache.org

Re: dynamic getTwitter ?

2016-08-25 Thread Andy LoPresto
ide> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Aug 25, 2016, at 11:48 AM, Andy LoPresto <alopre...@apache.org> wrote: > > Another crazy idea — would it be more computationally efficient to u

Re: dynamic getTwitter ?

2016-08-25 Thread Andy LoPresto
needs to be searched, duplicate an existing processor and replace the search term? They could all be located in a specific PG to allow for isolation from the “meta-flow” that is operating on NiFi itself. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5

Re: NiFi processor to convert CSV to XML

2016-08-25 Thread Andy LoPresto
://andrewjwelch.com/code/xslt/csv/csv-to-xml_v2.html <http://andrewjwelch.com/code/xslt/csv/csv-to-xml_v2.html> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Aug 25, 2016, at 3:16 PM, Nathamuni, Ramanujam

Re: NiFi Cluster mode

2016-09-27 Thread Andy LoPresto
-docs/html/administration-guide.html#configuration-best-practices Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Sep 27, 2016, at 11:27 AM, Manish Gupta 8 <mgupt...@sapient.com> wrote: > >

Re: read in values from nifi.properties in a Groovy ExecuteScript processor

2016-09-27 Thread Andy LoPresto
operations yourself. The framework abstracts this so it is not necessary for standard access. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Sep 27, 2016, at 5:36 AM, Tom Gullo <tomgu...@gmail.com> wrote:

Re: Frontend for Nifi support / mailing list

2016-10-29 Thread Andy LoPresto
<https://lists.apache.org/list.html?users@nifi.apache.org> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 27, 2016, at 6:19 AM, Davy De Waele <ddewa...@gmail.com> wrote: > > Oh ...

Re: UI is not opening after forming nifi 1.0.0 secure cluster in windows

2016-11-10 Thread Andy LoPresto
the connection below. $ openssl s_client -connect -debug -state -cert -key -CAfile Are there any errors in $NIFI_HOME/logs/nifi-app.log or $NIFI_HOME/logs/nifi-bootstrap.log? Are there any entries in $NIFI_HOME/logs/nifi-user.log? Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com

Re: What else besides port conflict will keep NiFi 1.0.0 from running?

2016-11-04 Thread Andy LoPresto
That’s a tricky one. Great catch Bryan — good news that it had gotten you before so you knew what it was, bad news that it happens to anyone. I think this something we can catch during bootstrap and provide a better error for. Russ, feel free to file a Jira for that. Andy LoPresto alopre

Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups

2016-10-20 Thread Andy LoPresto
ifi-1.0.0-secure-site-to-site> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 20, 2016, at 7:36 AM, Conrad Crampton <conrad.cramp...@secdata.com> > wrote: > > Ok, > So I tried

Re: Build failure under CentOS 6.7

2016-10-18 Thread Andy LoPresto
Thanks Michael. I'll review this when I am back at my keyboard. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 18, 2016, at 12:08, Andy LoPresto <alopre...@apache.org> wrote: > > Mic

Re: provenance & content repos re infosec

2016-10-20 Thread Andy LoPresto
your input on these feature development efforts. There are some other members of our community similarly security-minded, and I think we will get some great collaboration on this moving forward. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F

[DISCUSS] Security Feature Roadmap

2016-10-21 Thread Andy LoPresto
pache.org/confluence/display/NIFI/Security+Feature+Roadmap> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 signature.asc Description: Message signed with OpenPGP using GPGMail

Re: How to increase the processing speed of the ExtractText and ReplaceText Processor?

2016-10-24 Thread Andy LoPresto
Excellent information here. Thanks Lee and Peter. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 24, 2016, at 6:57 AM, Peter Wicks (pwicks) <pwi...@micron.com> wrote: > > Prabhu, >

Re: Build failure under CentOS 6.7

2016-10-18 Thread Andy LoPresto
No worries. Thanks for helping us track this down. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 18, 2016, at 1:07 PM, Giordano, Michael > <michael.giord...@vistronix.com> wrote: > >

Re: multiple flows and cluster

2016-11-25 Thread Andy LoPresto
your cluster into microclusters where each shares a partial flow (grouped by performance considerations or logical concepts). Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Nov 25, 2016, at 04:13, De Me

Re: Nifi Capability for Fast transfer of Data

2016-11-25 Thread Andy LoPresto
bly be the long pole in the tent here. Feel free to correct any poor assumptions or bad math above. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Nov 24, 2016, at 20:48, shweta Aggarwal <shweta.agg1...@gmail

Re: UI is not opening after forming nifi 1.0.0 secure cluster in windows

2016-11-11 Thread Andy LoPresto
ration-guide.html#tls-generation-toolkit <https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#tls-generation-toolkit> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Nov 10, 20

Re: Mark attribute as protected.

2016-10-11 Thread Andy LoPresto
feedback helps the developers prioritize what goes into future releases. [1] https://issues.apache.org/jira/browse/NIFI-1140 <https://issues.apache.org/jira/browse/NIFI-1140> [2] https://issues.apache.org/jira/secure/CreateIssue!default.jspa Andy LoPresto alopre...@apache.org alopres

Re: Best practices for pushing to production

2016-10-13 Thread Andy LoPresto
https://nifi.apache.org/docs/nifi-docs/rest-api/index.html Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 11, 2016, at 7:37 PM, Stéphane Maarek <stephane.maa...@gmail.com> > wrote: > > Hi, &g

Re: Rest API Client swagger.json

2016-10-13 Thread Andy LoPresto
as it was disabled by default. Not sure what others' feelings are. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Oct 13, 2016, at 5:13 PM, Matt Gilman <matt.c.gil...@gmail.com> wrote: > > Thanks

Re: How to log using logattribute processor

2017-01-10 Thread Andy LoPresto
want, and that flowfiles are being processed by the processor. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan 10, 2017, at 9:20 AM, Selvam Raman <sel...@gmail.com> wrote: > > Hi, > >

Re: Upgrade Nifi from 0.6 to 1.0

2016-12-03 Thread Andy LoPresto
Ali, NiFi version changes are intentionally designed to be backwards compatible when possible, and unless something in the migration notes specifically applies to your flow, you should be fine. You can copy the conf/flow.xml.gz file directly to the new instance. Andy LoPresto alopre

Re: Hanging on SplitJSON

2016-11-29 Thread Andy LoPresto
, given that you are more familiar with your specific problem space and the types of incoming data. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Nov 29, 2016, at 5:50 PM, Olav Jordens <ola

Re: Hanging on SplitJSON

2016-11-29 Thread Andy LoPresto
flowfile containing 100-1000 JSON objects and then route them to the SplitJSON processor. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Nov 29, 2016, at 5:34 PM, Olav Jordens <olav.jord...@2degreesmobile

Re: multiple flows and cluster

2016-12-05 Thread Andy LoPresto
Benoit, “Microclusters” is not a NiFi feature, it was just me saying you could logically separate flows which do not rely on each other into disparate clusters. So, for example, rather than running 1 12-node cluster, you could run 4 3-node clusters. Andy LoPresto alopre...@apache.org

Re: NiFi UI fails to display Processor Group: NullPointerException

2017-01-06 Thread Andy LoPresto
Would this be as simple as adding a version field (or hash of this and some other fairly static information) to the NodeHeartbeat interface? Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan 6, 2017, a

Re: HMAC key storage?

2016-12-18 Thread Andy LoPresto
whatever mechanism it exposes (Java API, PKCS#11, etc.). Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Dec 18, 2016, at 11:26, Jonathan Telfer <jntel...@gmail.com> wrote: > > Hi Simon > >

  1   2   3   4   5   >