Re: apache nifi 1.1.2 startup really slow on openstack environment

t; On Wed, Apr 5, 2017 at 1:07 PM, 尹文才 <batman...@gmail.com> wrote: > >> Thanks Andre, you mean the slow startup of NiFi secure cluster on >> openstack currently could be remedied by updating bootstrap.conf to >> instruct NiFi to use */dev/urandom, I will give it a try tom

Re: apache nifi 1.1.2 startup really slow on openstack environment

Wed, Apr 5, 2017 at 7:45 PM, Andre <andre-li...@fucs.org> wrote: > >> Hi, >> >> Can you please try the fix discussed here: >> >> https://issues.apache.org/jira/browse/NIFI-3313 >> >> Kind regards >> >> On Wed, Apr 5, 2017 at 7:36 PM, 尹文才

Re: disk space cleanup for running NIFI

ny errors in > the logs about the FlowFile Repo? > > Thanks > -Mark > > > On Jul 13, 2017, at 8:42 PM, 尹文才 <batman...@gmail.com> wrote: > > Thanks Russ, I checked your notes and found them very helpful. > Just like you I'm also using NIFI as a tool for ETL proc

Re: disk space cleanup for running NIFI

o you. > > http://www.javahotchocolate.com/notes/nifi.html#20170428 > > Hope this helps, > > Russ > > > On 07/12/2017 08:38 PM, 尹文才 wrote: > > Hi guys, I have a question about cleaning up the disk space used by NIFI > from time to time. > As you know NIF

disk space cleanup for running NIFI

Hi guys, I have a question about cleaning up the disk space used by NIFI from time to time. As you know NIFI saves a lot to disks, like the repository folders. I checked the NIFI official admin guide and I know the content repository supports toggling content archiving. So in order to save disk

Custom Processor UI guide

Hi guys, I'm trying to write a custom UI for my customized processor(The advanced page), I checked the official NIFI developer guide and only found a section called Custom Processor UIs which only briefly introduces about writing a Custom UI for a processor. Is there any tutorial on writing a

Re: How to wait for multiple processors' flowfile?

<psaltis.and...@gmail.com>: > Hi, > I think the new Wait / Notify processors, can be used to solve this > problem. Koji has a great blog post on it [1]. > > > [1] http://ijokarumawak.github.io/nifi/2017/02/02/nifi-notify-batch/ > > Thanks, > Andrew > > On Tu

Re: How to use wait\notify Processor?

Hi, I think you need to first setup the distributed cache service to make wait/notify processors work because wait/notify makes use of it to store signals. Regards, Ben 2017-05-11 20:29 GMT+08:00 prabhu Mahendran : > favorite >

How to wait for multiple processors' flowfile?

Hi guys, I'm currently using NiFi to do some ETL work and I need to handle the following problem: for example I have 3 self-defined processor A, B and C. all of them contain logic to incrementally update a database table. the table in C is created by table in A join table in B, so C depends on

how to set version for NIFI customized Processor?

Hi guys, I have installed NIFI 1.2 and have played with it for a while. One thing I noticed for 1.2 is that when I placed my previously written customized Processor into 1.2, I could see in the Processor select dialog there's a Version field for each processor and the value for my processor is

is it possible for a NIFI processor to be both timer driven and event driven?

Hi guys, actually I have 2 questions, what I need to do is like this: I have a Java program outside NIFI and inside NIFI I have a Customized processor inside NIFI, I need to send some data from my Java program to my NIFI processor and the processor is timer-driven by now. The processor needs to do

Re: how to set version for NIFI customized Processor?

t; > [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance > > Thanks > Joe > > On Wed, May 17, 2017 at 10:37 PM, 尹文才 <batman...@gmail.com> wrote: > > Hi guys, I have installed NIFI 1.2 and have played with it for a while. > > One thing I notic

add a property without validator in custom processor

Hi guys, a very simple question, I'm working on a customized processor and I'm trying to add a processor property without any validator and it prompted me that the property is not a supported property. Is there any way to add a property without any validator? Thanks. Ben, Regards

Extract data from Microsoft Access with NIFI Processors

Hi guys, currently I need to use NIFI to extract data of a Microsoft Access database, is there a currently available NIFI processor that can do this or do I have to write one for it? Thanks. Regards, Ben

Re: Extract data from Microsoft Access with NIFI Processors

th you go. > > Thanks > > On Wed, Jun 21, 2017 at 11:01 PM, 尹文才 <batman...@gmail.com> wrote: > > Hi guys, currently I need to use NIFI to extract data of a Microsoft > Access > > database, is there a currently available NIFI processor that can do this > or > > do I have to write one for it? Thanks. > > > > Regards, > > Ben >

get configured property values of DBCPConnectionPool controller service

Hi guys, I'm writing a customized processor that needs to use DBCPConnectionPool controller service, as you know database information like database url and driver name, etc are configured inside the properties of the DBCPConnectionPool Controller Service, is there a way inside my processor to get

NiFi's JdbcCommon convertToAvroStream problem

Hi guys, I'm currently writing a cusomized processor to try to extract data from a sqlite database and I have referenced some of the code from the NIFI QueryDatabaseTable processor, which uses the avro related Class JdbcCommon. I came across errors when converting the data in sqlite database into

Re: NiFi's JdbcCommon convertToAvroStream problem

; > Regards, > Matt > > [1] https://github.com/xerial/sqlite-jdbc/blob/3.8.11.2/src/ > main/java/org/sqlite/jdbc3/JDBC3ResultSet.java#L959 > [2] https://github.com/xerial/sqlite-jdbc/commit/ > 567b6da7616718c1f37ce70051e5570a42b58ea1 > > On Fri, Jun 23, 2017 at 11:56 PM, 尹文才 <

where is NIFI state data stored when in local mode?

Hi guys, does anyone know where NIFI stores the state data when I call statemanager.setState() if NIFI is run in local standalone mode? And is it OK to use statemanager.setState(Scope.CLUSTER) for both standalone and clustered NIFI? Thanks. Regards, Ben

Re: get configured property values of DBCPConnectionPool controller service

's probably possible to use reflection to access the values. > > You might be better off defining your own controller service interface > with an API that supports the information you want, and building an > implementation by inheriting from DBCPConnectionPool. > > Thanks, > > Jam

Extract data from one database into another one

Hi guys, currently I want to use NIFI processors to copy all tables and the data inside from one database(e.g. mysql) into another database(e.g. sql server). I know the processor QueryDatabaseTable could be used to extract data from a database and PutDatabaseRecord could be used to write avro data

Re: where is NIFI state data stored when in local mode?

ger > > NiFi Admin Guide - State Management > https://nifi.apache.org/docs/nifi-docs/html/administration-g > uide.html#state_management > > Thanks, > > James > > On Mon, Jun 26, 2017 at 8:53 PM, 尹文才 <batman...@gmail.com> wrote: > >> Hi guys, does anyone know

use DistributedMapCacheClientService in NIFI cluster

Hi guys, I'm currently using NIFI with 3 nodes as a cluster, when using DistributedMapCacheClientService there's a configuration property called 'Server Hostname'. I've tried with localhost on my local NIFI standlone node and it did work. My question is what should I set inside the NIFI cluster?

Re: use DistributedMapCacheClientService in NIFI cluster

l the clients they'll > only point to the server service on one of the nodes. > > Thanks > Joe > > On Thu, May 25, 2017 at 5:29 AM, 尹文才 <batman...@gmail.com> wrote: > > Hi guys, I'm currently using NIFI with 3 nodes as a cluster, when using > > DistributedMapCache

Re: use DistributedMapCacheClientService in NIFI cluster

full caching service. > In the next release you can script out a controller service as a cache > client. I believe one of the folks in the community has an example > floating around on how to do this to talk to Hazelcast. > > Thanks > Joe > > On Thu, May 25, 2017 at 8:45 PM, 尹文

Re: NIFI startup error

Sorry for that the reason why my NIFI didn't get up is not due to the warning listed above, but I still wonder why there're such warnings. Regards, Ben 2017-05-26 15:06 GMT+08:00 尹文才 <batman...@gmail.com>: > Hi guys, I'm currently writing a customized NIFI processor and after > put

NIFI startup error

Hi guys, I'm currently writing a customized NIFI processor and after putting the nar into the lib folder and tried to relaunch NIFI, it was giving a bunch of following errors: 2017-05-26 14:52:33,183 WARN [main] org.apache.nifi.nar.ExtensionManager Controller Service

Re: use DistributedMapCacheClientService in NIFI cluster

ould go for external > caching / databases (Redis?) > ZooKeeper is good for any HA / balanced use case, which does not have > heavy traffic, so if you have it installed and you know how to do it > in ZooKeeper, go for it. > > Regards, > Michal > > On Fri, May 26, 2017 at

Re: how to set version for NIFI customized Processor?

with the latest nifi nar maven plugin and it will get > > the version info at that time. > > > > thanks > > > > On Thu, May 18, 2017 at 4:20 AM, 尹文才 <batman...@gmail.com> wrote: > >> Thanks Joe, is it possible to set a specific version for a customized

Re: how to set version for NIFI customized Processor?

g this you should see three new entries in the MANIFEST file > that show the Nar-Dependency-Group, Nar-Dependency-Id, and > Nar-Dependency-Version. > > Let us know if this works. > > Thanks, > > Bryan > > On Sun, May 21, 2017 at 9:16 PM, 尹文才 <batman...@gmail.com> wrote:

Re: how to set version for NIFI customized Processor?

, and DBCPConnectionPool shows as 3.0 in NIFI. is there any way to work around this problem? Thanks. 2017-05-19 9:04 GMT+08:00 尹文才 <batman...@gmail.com>: > Thanks Bryan and Joe, I managed to set the specific version for my > processor with the properties. > > 2017-05-18 20:50 GMT

Re: how to set version for NIFI customized Processor?

nar/extensions/-unpacked/META-INF/MANIFEST.MF > > On Fri, May 19, 2017 at 3:16 AM, 尹文才 <batman...@gmail.com> wrote: > > Hi Bryan, I could set a specific version for my customized processor, but > > there's a side effect that the version of NIFI components I referenced &g

what data is transferred among nodes in a nifi cluster

Hi guys, I have written a custom processor trying to extract some data from a database and I'm currently using NiFi in standalone mode. I'm trying to migrate to use NiFi in clustered mode. However, I have a question about what data is transferred among the nodes in a cluster. For example, if I