Re: How to avoid this splitting of single line as multi lines in SplitText?

2017-02-14 Thread Andy LoPresto
Prabhu, I answered this on Stack Overflow [1] but I think you could do it with ReplaceText before the SplitText using a regex like "(.*?)(\n)(.*?)" replaced with "$1\\n$3" [1] http://stackoverflow.com/a/42242665/70465 Andy LoPresto

Re: How to avoid this splitting of single line as multi lines in SplitText?

2017-02-14 Thread Lee Laim
Prabhu, You need to remove the new lines from within the last field. I'd recommend using awk in an execute stream command processor first, then splitting the text. Alternatively, you could write a custom processor to specifically handle the incoming data. Lee > On Feb 14, 2017, at 11:01

Re: NIFI

2017-02-14 Thread Chris Herrera
Try that again sending from the right address this time :). > On Feb 14, 2017, at 9:40 PM, Chris Herrera > wrote: > > Hi Marcio, > > I played around with this a bit and eventually needed to build a custom > processor that essentially used thrift as a dependancy.

Re: NIFI

2017-02-14 Thread Andy LoPresto
Hello Márcio, I have not used Thrift, but in reading the introduction from the link you provided, my first reactions would be as follows: 1. To communicate with a service defined by Thrift, I’d look at InvokeHTTP and see if it covers your needs. 2. If not, I would try downloading the Thrift

Re: NiFi Users: Powered by NiFi page

2017-02-14 Thread Joe Witt
Corey, Hammad, Jeremy I've added your information to the powered-by page. Great info - thanks! All, keep em coming. This is really helpful for other users. Joe On Tue, Feb 14, 2017 at 7:36 PM, Corey Flowers wrote: > Name: Onyx Point > > Web: www.onyxpoint.com > >

Re: NiFi Users: Powered by NiFi page

2017-02-14 Thread Corey Flowers
Name: Onyx Point Web: www.onyxpoint.com Industry: Commercial/Federal large scale data distribution system support and design Description: design large scale NIFI clusters for high volume ingest/egress and provide day to day operational support and maintenance. Sent from my iPhone > On Feb

Re: NiFi Users: Powered by NiFi page

2017-02-14 Thread Andy LoPresto
Thank you Jeremy and Hammad. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 14, 2017, at 2:37 PM, Hammad wrote: > > Hi Joe, > Here you go. > > Name: Flexilogix > Industry:

Re: NiFi Users: Powered by NiFi page

2017-02-14 Thread Hammad
Hi Joe, Here you go. *Name*: Flexilogix *Industry*: BigData/IoT *Description*: Builds all data ingestion pipelines using NiFi. Have deployed NiFi clusters to ingest, transform and deliver to data analytics backends serving all purposes of data mediation both for realtime and batch jobs. Thanks

Re: Integration between Apache NiFi and Parquet or Workaround?

2017-02-14 Thread Bryan Bende
I'll caveat this by saying that up until 10 mins ago I had never looked at Parquet, so I could be completely wrong, but... The Parquet API seems heavily geared towards HDFS. For example, take the AvroParquetWriter:

Re: NiFi Users: Powered by NiFi page

2017-02-14 Thread Jeremy Farbota
Payoff (payoff.com) Financial Wellness (fintech) NiFi primarily serves as our consumer between Kafka and HDFS. NiFi also provides schema validation for event streams while enabling us to modify and republish secure event streams for general use. NiFi extracts and standardizes large datasets from

Re: Integration between Apache NiFi and Parquet or Workaround?

2017-02-14 Thread Carlos Paradis
Hi James, Thank you for pointing the issue out! :-) I wanted to point out another alternative solution to Kite I observed, to hear if you had any insight on this approach too if you don't mind. When I saw a presentation of Ni-Fi and Parquet being used in a guest project, although not many

Re: Integration between Apache NiFi and Parquet or Workaround?

2017-02-14 Thread James Wing
Carlos, Welcome to NiFi! I believe the Kite dataset is currently the most direct, built-in solution for writing Parquet files from NiFi. I'm not an expert on Parquet, but I understand columnar formats like Parquet and ORC are not easily written to in the incremental, streaming fashion that NiFi

Re: NiFi Users: Powered by NiFi page

2017-02-14 Thread Joe Witt
Could you list 'company/organization name', 'industry', and a description of how NiFi is used please. Thanks Joe On Tue, Feb 14, 2017 at 3:12 PM, Corey Flowers wrote: > Hey Joe, > > You can put us on their. Do you need a write up or anything about how > we are

Re: NiFi Users: Powered by NiFi page

2017-02-14 Thread Corey Flowers
Hey Joe, You can put us on their. Do you need a write up or anything about how we are using it? Corey Sent from my iPhone > On Feb 14, 2017, at 2:07 PM, Joe Witt wrote: > > NiFi Users > > I just realized we have a 'powered by nifi' page. It looks a >

Re: NiFi Users: Powered by NiFi page

2017-02-14 Thread Joe Witt
Hammad Excellent. Can you describe the 'industry' Flexilogix is a part of and a 'description' of its relationship/use for Apache NiFi? https://nifi.apache.org/powered-by-nifi.html Thanks Joe On Tue, Feb 14, 2017 at 2:54 PM, Hammad wrote: > Hi Joe, > > That would be nice.

Re: NiFi Users: Powered by NiFi page

2017-02-14 Thread Hammad
Hi Joe, That would be nice. Please mention 'FLEXILOGIX' [http://www.flexilogix.com] Thanks, Hammad On Tue, Feb 14, 2017 at 8:07 PM, Joe Witt wrote: > NiFi Users > > I just realized we have a 'powered by nifi' page. It looks a > little...light :-). So wanted to reach out

NiFi Users: Powered by NiFi page

2017-02-14 Thread Joe Witt
NiFi Users I just realized we have a 'powered by nifi' page. It looks a little...light :-). So wanted to reach out and offer to anyone interested that if you reply back on this thread with your company/organization that you'd like referenced on there I'd be happy to put in the change to the

RE: RemoveDistributedMapCache

2017-02-14 Thread Carlos Manuel Fernandes (DSI)
Matt, Yes, i have the "duplicate" relationship being routed back to the DetectDuplicate. Yes I set Max Concurrent Tasks in IncrementalLoadData to something greater than 1, permitting processing tables in "parallel". I Want process many flows with a specific tableName since I don’t