GenerateTableFetch Segment Identifier

2018-10-31 Thread Shawn Weeks
Currently GenerateTableFetch doesn't set any attributes to identify segments and unique executions like UnpackContent does. I was wondering if anyone else thought that might be useful or had another way to figure out this information. I need to be able to track that all segments have succeeded

Re: PutHiveStreaming TimelineClientImpl Exception

2018-10-31 Thread Shawn Weeks
You have to either create a hive-site.xml just for NiFi without the hook or your yarn-site.xml needs to be in the class path. Another parameter that you might have to set to make Hive streaming less chatty is hcatalog.hive.client.cache.disabled=true, it was recomened by our vendor to get rid

Re: GenerateTableFetch Segment Identifier

2018-10-31 Thread Matt Burgess
I’m not at my computer at the moment but I wrote 2 Jiras and put up 2 PRs to add these features, hopefully they will make the next release. I can get the links later today. > On Oct 31, 2018, at 10:27 AM, Shawn Weeks wrote: > > Currently GenerateTableFetch doesn't set any attributes to

Expression Language

2018-10-31 Thread Jones, Patrick L.
Howdy, I'm having trouble with the expression language. I have 2 attributes ElasticIP and ElasticPort. I'm trying to use them in an Elastic URL. If I hard code the values in it works find. If I use the attributes I get a null pointer. I have tried various variations on:

Re: Expression Language

2018-10-31 Thread Boris Tyukin
looks right to me...Did you check flowFile attributes in provenance events to make sure your attributes are populated? also check exact spelling and casing. If still does not work, show us some screenshots of your flow and properties Boris On Wed, Oct 31, 2018 at 4:04 PM Jones, Patrick L. wrote:

Re: Expression Language

2018-10-31 Thread Bryan Bende
You haven't said which processor/service you are using, but you may want to check the docs for that component to see whether it supports expression language from flow file attributes, some only support variable registry. Most of the elastic search processors look like the host only supports

Fwd: NIFI Usage for Data Transformation

2018-10-31 Thread Ameer Mawia
We have a use case where we take data from a source(text data in csv format), do transformation and manipulation of textual record, and output the data in another (csv)format. This is being done by a Java based custom framework, written specifically for this *transformation* piece. Recently as

Re: NIFI Usage for Data Transformation

2018-10-31 Thread Mark Rachelski
Ameer, Allow me to provide an opinion on this as a user, not as one of the awesome guys in this group that has built this very cool tool. They are likely to be very enthusiastic. In my experience, NiFi is not going to easily replace a custom tool that has done a number of complex transforms. For

PutHiveStreaming TimelineClientImpl Exception

2018-10-31 Thread Noe Detore
Hello, Using NIFI 1.5 PutHiveStreaming processor I am seeing a lot of logs INFO [ATS Logger 0] o.a.h.y.c.api.impl.TimelineClientImpl Exception caught by TimelineClientConnectionRetry, will try 1 more time(s). Message: java.net.ConnectException: Connection refused 2018-10-31 07:44:51,612 WARN