Re: How to control NiFi logging

2016-06-07 Thread Stéphane Maarek
Hi Andrew, It looks like the settings don't work. only the main logs go to the indicated folder, the other dated files still go to my internal log folder... ls /mnt/xvdf/logs nifi-app.log nifi-bootstrap.log nifi-user.log ls ~/nifi/logs/ nifi-app_2016-06-06_16.0.log

Re: How to control NiFi logging

2016-06-07 Thread Stéphane Maarek
fi-framework/nifi-resources/src/main/resources/conf/logback.xml > > Cheers > > On Wed, Jun 8, 2016 at 10:22 AM, Stéphane Maarek < > stephane.maa...@gmail.com> wrote: > >> Hi Andrew, >> >> It looks like the settings don't work. only the main logs go to the >> in

Re: How to control NiFi logging

2016-06-06 Thread Stéphane Maarek
ould want to change the references above in to your attached volume. > I have marked them with bold above. In the logback config file you will see > two other appenders as well, these same changes should be done there as > well. > > Thanks, > Andrew > > > On Mon, Jun 6

Guidance for NiFi output streaming

2016-05-25 Thread Stéphane Maarek
Hi, I need to output some data streaming from multiple devices directly into a map (mapboxjs). Basically, every 1 second, I want to only write the last data point for each device to a json file. My problem resides in "how to pick the latest data point by device" My incoming flow file has three

How to use GetMongo Projection

2016-05-26 Thread Stéphane Maarek
Hi, How do I construct a bson from the getMongo processor? Basically, I want to retrieve field_1, field_2 Side note: I think your overall documentation is great, but it's missing "example values". This will users figure out much more quickly what values is expected from properties Cheers

Re: Doing development on nifi

2016-04-29 Thread Stéphane Maarek
, > then when you start it, it will likely only bring in one flow file, so > you can react to how it works, then stop it before it empties the > queue. > > I hope that makes sense, I apologize in advance if I made things more > confusing. The good news is there is a solution to your

Apache NiFi with Fail2Ban

2016-04-27 Thread Stéphane Maarek
Hi, I think I have messed something up and I need some help I downloaded and untar nifi on my raspberry pi. Then I installed the service which is working (I can access nifi through my browser) Now something I can't do is the following: sudo apt-get install fail2ban How can I cleanly remove the

Doing development on nifi

2016-04-28 Thread Stéphane Maarek
Hi, I'm very new to nifi and love the concept. As part of the process, I'm learning. My biggest frustration is that I can't see the data flowing through the system as I do development. Maybe I missed an article or a link, but is it possible to view the data while in the flow? I.e. Say I create a

Re: ReplaceText Infinite loop?

2016-05-23 Thread Stéphane Maarek
> On May 23, 2016, at 20:08, Stéphane Maarek <stephane.maa...@gmail.com> > wrote: > > An easy way to reproduce is to take the csv to json template here: > > https://cwiki.apache.org/confluence/download/attachments/57904847/CsvToJSON.xml?version=1=1442927496000=v2 > >

Re: How to deal with decimals while they're not supported?

2016-08-12 Thread Stéphane Maarek
iously have to revert back to number to do the actual > comparison though. > > Haven’t tested but something like… > > myAttr:toString():substringAfter(‘.’):toNumber():lt(1) > > Conrad > > > > *From: *Stéphane Maarek <stephane.maa...@gmail.com> > *Reply-To: *&

Re: Increase the number of threads available to a Node

2016-07-19 Thread Stéphane Maarek
D3C4 BACE 3C6E F65B 2F7D EF69 > > On Jul 18, 2016, at 10:24 PM, Stéphane Maarek <stephane.maa...@gmail.com> > wrote: > > Hi Joe, > > Been using NiFi for a while and I have always been wondering why I was > capping at 10 thread. Now I have my answer :) > It would be

Re: Create a PutRethinkDB processor

2016-07-10 Thread Stéphane Maarek
/jruby/wiki/JRuby-Reference#java_annotation > [4] https://issues.apache.org/jira/browse/NIFI-2215 > > On Sun, Jul 10, 2016 at 7:42 PM, Stéphane Maarek > <stephane.maa...@gmail.com> wrote: > > Hi, > > > > I've been thinking about implementing a RethinkDB processor as I'

RouteText questions (regex, grouping, performance)

2016-06-30 Thread Stéphane Maarek
Hi, I have a question regarding RouteText. The processor works just fine for me but maybe I'm missing a couple subtleties: 1) I have a regex to group data by (a pair of IDs), but what do I use the grouping attribute for? I still get as many outputs as lines 2) My data is coming from a listenUDP.

Re: Java Heap Size increasing without anything happening

2016-06-30 Thread Stéphane Maarek
Repository, keeping > stats for all of the processors, etc., etc. > > If you were to see "Full Garbage Collections" taking place then there may > be some concern but minor garbage collections > are pretty frequent and shouldn't cause concern. > > Thanks > -Mark >

Best EC2 instance type for NiFi

2016-06-29 Thread Stéphane Maarek
Hi, I'm wondering which instance on AWS EC2 is best suited for NiFi (let's say for a standalone). I'm wondering if it's a compute instance (c4), or something else? and why? Thanks for your help! Stephane

Java Heap Size increasing without anything happening

2016-06-29 Thread Stéphane Maarek
Heap (32.0%) *Max* 5 GB *Total* 4 GB *Used* 1.62 GB *Free* 2.38 GB dd I have an issue, I followed this link to "optimize" HDF https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.html Basically, I have two UDP receivers but they don't receive any

Re: Rest API Client swagger.json

2016-10-13 Thread Stéphane Maarek
Investigated some more, open a JIRA issue, closed it via https://github.com/apache/nifi/pull/1135 On Fri, Oct 14, 2016 at 9:47 AM Stéphane Maarek <stephane.maa...@gmail.com> wrote: > Hi, > > Thanks it helps ! Good to know there is already a java client I could use. > Nonetheles

Docker to deploy NiFi

2016-10-11 Thread Stéphane Maarek
Hi, Has anyone deployed NiFi using docker? I see many images on dockerhub, but I have a few questions regarding usage: 1) How do you handle overall configuration (java memory options, etc) ? 2) If you add a docker process, does the spun up NiFi get automatically added to your cluster? 3) How do