Re: ExecuteProcess processor with TLS1.2 error: "failed setting cipher list"

/Him > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On May 22, 2020, at 8:08 AM, Eric Chaves wrote: > > Hi folks, > > I have a flow that downloads files from an FTP server over SSL using > TLS1.2. To achieve this I use curl command line in an Exe

Re: How write multiple prepends/appens in a single EL expression?

Thanks! Em qui., 19 de mar. de 2020 às 17:29, Paul Parker escreveu: > You can go without append and prepend. > > sftp://${sftp.remote.host}/${path}/${filename} > > Give it a try. > > Eric Chaves schrieb am Do., 19. März 2020, 21:05: > >> Hi folks, >>

How write multiple prepends/appens in a single EL expression?

Hi folks, Sorry for another newbie question. =) I'm trying to write a single EL expression to perform multiples string manipulation but I keep hitting an invalid EL expression. What would be the correct way to write this expression:

Re: How to iterate over all dynamic properties in python invokeScriptedProcessor?

ache.org > *alopresto.apa...@gmail.com * > He/Him > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Mar 18, 2020, at 12:12 PM, Eric Chaves wrote: > > Hi folks, > > I'm trying to write a quick python InvokeProcessorScript where I need to > itera

How to iterate over all dynamic properties in python invokeScriptedProcessor?

Hi folks, I'm trying to write a quick python InvokeProcessorScript where I need to iterate over all Dynamic Properties from the processor to select just a few and I'm having some difficulties with the class types between Jython and Java. Can someone show me how to iterate over

Re: Can someone confirm if bug NIFI-5830 is fixed on docker hub images

ything we should rethink on this front to help > alleviate such issues and ease transitions. > > aldrin > > On Mon, Mar 16, 2020 at 9:46 AM Eric Chaves wrote: > >> Hi Aldrin, >> >> Ok, I think I got my error. I'm using a docker-compose to run a local >>

Re: Can someone confirm if bug NIFI-5830 is fixed on docker hub images

dalone NiFi instance? > How are you running your Docker containers? Do you have these on the same > Docker network? If you are using compose, this should be taken care of by > default, and if so, could you please share your compose file? > > > > > > On Sun, Mar 15, 2020 a

Can someone confirm if bug NIFI-5830 is fixed on docker hub images

Hi folks, I'm facing an error using Redis DMC in standalone mode with an external Redis server were the RedisConnectionPoolService is not able to connect to a non-localhost redis in this mode. This error was already reported ( https://issues.apache.org/jira/browse/NIFI-5830) and marked fixed in

Re: What UDFs are supported by QueryRecord?

/1.9.2/org.apache.nifi.processors.standard.QueryRecord/index.html > > On Oct 16, 2019, at 2:10 PM, Eric Chaves wrote: > > Ok, figured out a way to do it. I noticed that QueryRecord uses Apache > Calcite so I tried some of Calcite functions until I got into this > statement that seems

Re: What UDFs are supported by QueryRecord?

ill like to know if (and how) I could use either NiFi expression or RecordPath functions in a QueryRecord statement. Thanks in advance, Em qua, 16 de out de 2019 às 14:30, Eric Chaves escreveu: > Hi Folks, > > I'd like to use a SQL statement in QueryRecords like this uses the extract &

What UDFs are supported by QueryRecord?

Hi Folks, I'd like to use a SQL statement in QueryRecords like this uses the extract the last 10 chars of field and compare it to today, similar to the line below: SELECT substring( '/index', 10, -1) as expired FROM FLOWFILE WHERE expired <= '${now():format("-MM-dd")}' This statement is

Re: Which JAR can I find org.apache.nifi.record.path.*

AR, which is not > currently included in the scripting bundle. I have written up an > improvement Jira [1] to add this. > > Regards, > Matt > > [1] https://issues.apache.org/jira/browse/NIFI-6741 > > On Fri, Oct 4, 2019 at 3:30 PM Eric Chaves wrote: > > > > H

Which JAR can I find org.apache.nifi.record.path.*

Hi folks, I'd like to use the Record Path classes in a groovy script for a InvokeScriptedProcessor but when I import those classes they aren't found. Can anyone point me the JAR where they reside? The classes I'm importing are import org.apache.nifi.record.path.FieldValue import

Re: Validate CSV/Records by name instead of position

; On Wed, Sep 18, 2019 at 9:59 AM Eric Chaves wrote: > >> Hi folks, >> >> Is it possible to validate fields/columns in Record or CSV by its name >> instead of it's position? For example I have a record with two mandatory >> fields and some optional fields but they m

Validate CSV/Records by name instead of position

Hi folks, Is it possible to validate fields/columns in Record or CSV by its name instead of it's position? For example I have a record with two mandatory fields and some optional fields but they may be on different position on each ingested file. Should I use a script or there is already a

Re: Redis - Sorted Sets

Hi John, Here at work we also needed to use redis in a flow to store and retrieve real-time events temporary. We haven't found any "Put/Fetch Redis" processors and the RedisDistributedMapCache didn't fit out needs precisely because our flow required more control over the key namespace and also to

Re: How to lock mysql a table for concurrent reads

Hi folks, found my error. I fixed my query to use an inner join with a select for update instead of a subselect and configured the PutSQL to use a batch count of 1. Thanks! Em qui, 12 de set de 2019 às 19:10, Eric Chaves escreveu: > Hi folks, > > I'm having a hard time to figure out

How to lock mysql a table for concurrent reads

Hi folks, I'm having a hard time to figure out how to do an update inside an atomic transaction with NiFi. In MySQL I use a table to control what is kind of a "record counter" distributed among files. The table looks like this: *create table file_records (* * file_id BIGINT NOT NULL

Can FTP processors ignore server internal ftp address in passive mode?

Hi folks, My nifi flows are facing a trouble with some FTPS servers who send their internal ip address in passive mode and therefore the FTP processor cannot open the second connection for data transfer. eg.: *Command: PASV* *Response: 227 Entering Passive Mode (172,17,4,21,120,207).* Is it

Re: Advice on orchestrating Nifi with dockerized external services

possible approach > A). > https://github.com/apache/nifi-minifi-cpp/tree/master/nanofi > > The latest MiNiFi (C++) version 0.6.0 was released recently. > https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes > > Thanks, > Koji > > On Thu, Apr 11, 2019 at 2:28 AM Eric

Advice on orchestrating Nifi with dockerized external services

Hi Folks, My company is using nifi to perform several data-flow process and now we received a requirement to do some fairly complex ETL over large files. To process those files we have some proprietary applications (mostly written in phyton or go) that ran as docker containers. I don't think

Re: Clarifications on getting flowfiles using FlowFileFilters

ons to see if you can narrow down what's going > on there. Also, what content repository implementation are you using? > I think there's a new one that's not yet the default but works better. > > Regards, > Matt > > On Wed, Mar 27, 2019 at 3:12 PM Eric Chaves wrote: > > >

Clarifications on getting flowfiles using FlowFileFilters

Hi folks, I'd like to write InvokeScriptedProcessor that inspect all files in the incoming queue without actually processing/transfering them until a given business condition is met. So I looked for some examples of how could I get all flowfiles in a queue using session.get(FlowFileFilter) and I

Re: How can I ExtractGrok from end-of-file?

g creative > probably using RouteOnAttribute, GrokExtract, MergeContent (with > defragment merge strategy) ... etc. > > Thanks, > Koji > > On Fri, Mar 15, 2019 at 11:34 PM Eric Chaves wrote: > > > > Hi folks, > > > > I'm learning how to use grok with n

Can't trim leading whitespaces with UpdateRecord

Hi Folks, Forgive me for this dummy question but I've been cracking my head for hours. Is there anything special in RecordPath regular expressions? I'm trying to trim whitespaces by using UpdateRecord (with replacement strategy of Record Path Value ) replaceRegex over current field value but I

Is it possible to use declare an Avro schema for multi-record files?

Hi folks, Is possible to declare an Avro schema for a ConvertRecord processor to handle multi-record file ie a file where each line may be a different avro record? Something like this: { "type" : "record", "namespace" : "com.acme", "name" : "OrderFile", "fields" : [ {

Export/Import flow files

Hi fellows, I'd like to export a flow file from a queue running in production in order to import it into my local machine for development/troubleshooting? Also, is it possible to pick a single flow file in a given queue and process only it? Is there any way to do it? Best regards, Eric

How can I ExtractGrok from end-of-file?

Hi folks, I'm learning how to use grok with nifi starting with the ExtractGrok processor and I'd like to use it to extract data from file headers and trailers however since the GrokExtract processor only apply the grok expression on the defined buffer size (and each of my file differs on size) I

Re: Help with SSL Context Service for https post requests

? > > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Dec 8, 2017, at 2:11 PM, Eric Chaves <e...@uolet.com> wrote: > > Hi Andy, >

Re: Help with SSL Context Service for https post requests

CE 3C6E F65B 2F7D EF69 > > On Dec 8, 2017, at 10:21 AM, Eric Chaves <e...@uolet.com> wrote: > > Hi, > > I'd like to make an HTTPS request to an internet public service but I'm > failing to to setup the SSL Context Service. I tried to export my system > certs to be used as t

Help with SSL Context Service for https post requests

Hi, I'd like to make an HTTPS request to an internet public service but I'm failing to to setup the SSL Context Service. I tried to export my system certs to be used as truststore. openssl pkcs12 -export -nokeys -in /etc/ssl/certs/ca-certificates.crt -out ./assets/truststore.p12 Can someone

Re: How can I change the content-type of a flow file in a script processor?

> Russ > > On 12/07/2017 07:12 PM, Chris Herrera wrote: > > Hi Eric, > > If its for the content viewer you might want to use mime.type. > additionally you can use session.putAttribute(flowFile, “content-type”, > "application/json”); > > Regards, > Chris > > &

How can I change the content-type of a flow file in a script processor?

Hi guys, I'm trying to programatically set the content-type of a flow-file by calling flowFile.putAttribute('content-type', 'application/json') but it's not working. How can I change my flow-file content-type? Regards, Eric

Re: Someone recommend a good Avro studdy guide for newbie?

b.com/apache/nifi/blob/master/nifi-nar- > bundles/nifi-standard-bundle/nifi-standard-processors/src/ > main/java/org/apache/nifi/processors/standard/ > AbstractRecordProcessor.java#L94 > > > On Wed, Nov 22, 2017 at 3:00 PM, Eric Chaves <e...@uolet.com> wrote: > > Hi

Someone recommend a good Avro studdy guide for newbie?

Hi folks, Due to my lack of knowledge with Avro in my scripted processors I'm required to convert them to JSON in order to be able to manipulate them. I would like to avoid this and write record based processors but I have no clue about how should I do to actually read/write/modify record based

Is it possible to import class from NAR bundle in scripted processor?

Hi folks, I'm writing some groovy scripts for that will use the AWS-SDK. My first shot was to write a InvokeScriptedProcessor that extends the AbstractAWSCredentialsProviderProcessor but the script is unable to resolve this class. Is it possible to reference a class in a NAR bundle? Regards,

Re: DBCP Error Cannot get a connection, pool error Timeout waiting for idle object

as expected. I'm just finishing some rough points and once I finish it I'll share it with everyone in a gist. Thanks everyone so far! =) 2017-11-16 13:49 GMT-02:00 Eric Chaves <e...@uolet.com>: > Hi guys, > > I've made a lot of changes on my script processor and now I'm pro

DBCP Error Cannot get a connection, pool error Timeout waiting for idle object

Hi guys, I've made a lot of changes on my script processor and now I'm properly getting an instance of DBCPService however when I try to use the connection I got the error org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object. I know the

Re: How to get DBCP service inside ScriptedLookupService

What is my mistake here? Cheers, Eric 2017-11-15 12:24 GMT-02:00 Eric Chaves <e...@uolet.com>: > Matt, Mark, thanks for the great explanations! I'm learning a lot! :) > > So I went down the road described but I'm getting another error: >

Re: How to get DBCP service inside ScriptedLookupService

but > >> IIRC only works if you don't have a security manager/policies on the > >> JVM. > >> > >> TL;DR You can't currently get controller services by name in the > >> initialize() method, you have to implement onEnabled instead. If you > >> want to use logg

How to get DBCP service inside ScriptedLookupService

Hi Folks, I need to get an instance of DBCPService inside my ScriptedLookupService and for that I'm following Matt's post http://funnifi.blogspot.com.br/2016/04/sql-in-nifi-with-executescript.html In my groovy class I've overrided the initialize method and performing the lookup there but I'm

Re: How to make a lookup service return a Record

ifi-standard-services/nifi- > mongodb-services-bundle/nifi-mongodb-services/src/main/ > java/org/apache/nifi/mongodb/MongoDBLookupService.java > > On Sun, Nov 12, 2017 at 4:45 PM, Eric Chaves <e...@uolet.com> wrote: > >> Hi Folks, >> >> I've being cracking

How to make a lookup service return a Record

Hi Folks, I've being cracking my head before asking but couldn't figure out how to make an ScriptedLookupService (groovy) return either a Record or an Map. So far I was only able to return single values. Can anyone point me in the right direction? Cheers,

Re: Enrichment

Hi Rahul, I'm working something similar and my first shoot was writing a Scriptedookup service to perform database lookups. Not sure if this is the recommended approach but it is working. In case you want to give it a try you can take a look on Matt's articles "Scripting Cookbooks" to get a hang

Re: Enrichment flow using ScriptedLookup

exists and update the level, or add a node if it does not, then writes > it back out to logback.xml). > > Regards, > Matt > > > > On Sun, Nov 5, 2017 at 9:45 AM, Joe Witt <joe.w...@gmail.com> wrote: > > eric > > > > can you please file a JIRA to r

Re: Enrichment flow using ScriptedLookup

disable) to get that working. > > On Sun, Nov 5, 2017 at 8:54 AM, Eric Chaves <e...@uolet.com> wrote: > >> Hi Mike, I'm running nifi using the official docker image (version 1.4.0) >> and my logs folder is empty. I was looking at bootstrap.conf, logback.xml >> an

Re: Enrichment flow using ScriptedLookup

Mike Thomsen <mikerthom...@gmail.com>: > You may need to update the logback xml file in the conf folder. There is a > line in there for the processor package. Might be too high for info. > > On Sat, Nov 4, 2017 at 10:50 AM Eric Chaves <e...@uolet.com> wrote: > >> Hi

What would be a good way to build NoSQL entity from SQL data?

Hi fellows, I'm planning a flow to feed data from a relational database into elasticsearch and would like to ask for some advice. The database has an entity Person with 1:N relation to Email and Phone (email and phone are objects, not just scalar values). The flow needs to feed ES with JSON

Re: How to move queue boxes

the black line of the > relationship (and remove a bending point by double clicking on it). > > Pierre > > 2017-10-23 11:38 GMT+02:00 Eric Chaves <e...@uolet.com>: > >> Hi, my flow has a lot processors coming out from a single router >> processor which makes the diagram

Re: How to move queue boxes

relationship (and remove a bending point by double clicking on it). > > Pierre > > 2017-10-23 11:38 GMT+02:00 Eric Chaves <e...@uolet.com>: > >> Hi, my flow has a lot processors coming out from a single router >> processor which makes the diagram a mess with queue

How to move queue boxes

Hi, my flow has a lot processors coming out from a single router processor which makes the diagram a mess with queue's boxes overlapping each other. sometimes when I connect two processors the queue box connecting them has a small yellow square that allows moving the box around producing angled

Re: Advice on sending high volumes of email with putEmail

itt <joe.w...@gmail.com>: > that rate should be perfectly fine on a modest host system. > > On Oct 21, 2017 12:40 PM, "Eric Chaves" <e...@uolet.com> wrote: > >> Hi Folks, >> >> I'm planning to use nifi to send transactional e-mails with PDF attached. &g

Help with UnsupportedDataTypeException: no object DCH for MIME type

Hi folks, after some trial and error (I'm not a java developer) I think I managed to write a groovy script to build a multi mimepart message, but now I'm stuck in an exception that I have no clue about what to do: "javax.activation.UnsupportedDataTypeException: no object DCH for MIME type

ImportingCan I import javax in a script processor?

Hi folks, I'm writing a script processor to send email messages with an attachment. After reading the putEmail processor and googling a little bit I came up with the attached script however my script throws exceptions when importing any javax.mail classes (ex.: unable to resolve class

Formatting string numbers and date

Hi, is it possible to format numbers and dates with NiFi's Expression Language? Best regards,

Advice on sending high volumes of email with putEmail

Hi Folks, I'm planning to use nifi to send transactional e-mails with PDF attached. I'll be dispatching around 10K messages per day with peaks up to 1k per hour. Do you think nifi can handle this task? Also, any advices on sizing and dimension of the nifi node for this task? Thanks in advance,

Re: Parsing stringified json content

Message" content with > EvaluateJsonPath by setting the Destination property to flowfile-content. > > Thanks, > > James > > > On Sun, Oct 8, 2017 at 3:38 PM, Eric Chaves <e...@uolet.com> wrote: > >> Hi, I'm writing a flow to handle SNS notifica

Parsing stringified json content

Hi, I'm writing a flow to handle SNS notification messages received through HTTP. Those messages are JSON content whose Message attribute is an stringified JSON text. How can I parse this content into a new flow file? Thanks in advance, Eric

Re: tips for flow development and version control

i.apache.org/docs/nifi-docs/html/user-guide.html#templates > [2] https://cwiki.apache.org/confluence/display/NIFI/ > Configuration+Management+of+Flows > [3] https://issues.apache.org/jira/projects/NIFIREG > > Thanks > Joe > > On Fri, Oct 6, 2017 at 7:55 AM, Eric Chaves <

tips for flow development and version control

Hi folks, I'm starting with nifi and would like to ask for some tips regarding how to best setup/manage the a project folder with multiple flows using git. Do you have any tips to share? Cheers, Eric