Re: [DISCUSS] Apache NiFi distribution has grown too large

Just an idea, can we also manger Nar, custom Nars in NiFi Repository and let fresh NiFi installers to add required nars and their dependencies on the fly vi NiFi UI ? -Sumanth > On Jan 17, 2018, at 8:05 AM, Matt Burgess wrote: > > I'd like to echo many of the comments

VolatileProvenanceRepository for MiNiFi

Hi, In NiFi , I can set following properties in 'nifi.property'. nifi.provenance.repository.implementation=org.apache.nifi.provenance.VolatileProvenanceRepository nifi.provenance.repository.buffer.size=1000 What is the equivalent process for MiNiFi configuration ? -- Thanks, Sumanth

Re: MiNiFi logs not removed as per l maxHistory in ogback.xml

rently under vote (0.2.0) should have these >> issues resolved since including newer releases of the logback lib. If you >> would be able to evaluate please let us know if there continue to be issues. >> >>> On May 15, 2017, at 13:43, Sumanth Chinthagunta <xmlk...@gmail

MiNiFi logs not removed as per l maxHistory in ogback.xml

We have an issue with MiNiFi (minifi-0.1.0 Java). It is not rotating its logs as per conf/logback.xml maxHistory value. Over the time, it is filling the drive and leaving many log files on desk. many "minifi-app.log15558502285619089.tmp" files are left on desk. please advice. Thank

TailFile Multiple files and Rolling Filename Pattern not working.

Hi I am trying to use TailFile Processor to aggregate logs. my app produce three log files : 1. app-sumo-audit.log 2. app-sumo-api.log 3. app-sumo-logging.log and corresponding rolling files app-sumo-audit.log.2017-03-11.0 app-sumo-audit.log.2017-03-11.1 app-sumo-api.log.2017-03-11.0

Re: Odd inter-operation behavior when using clustered NiFi against secure MapR cluster

Andre, Sorry for late replay. Yes we are able to successfully integrate nifi with secure MapR cluster with JAAS method Sumo Sent from my iPhone > On Oct 27, 2016, at 1:38 AM, Andre wrote: > > Sumo, > > Have you tried the workaround? Did it work? > > Cheers > >> On

Re: DB2 to MS SQL data transfer

rt the source table to a file, then > transfer it with NiFi to the target system, then use PutSQL to do a > BULK INSERT [2]. > > Regards, > Matt > > [1] > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.QueryDatabaseTable/index.html >

Re: DB2 to MS SQL data transfer

gt; wrote: > > Hello Sumo, > > Are you looking at transferring data from a specific table between two > databases, or all tables? > > On Wed, Aug 10, 2016 at 3:17 PM Sumanth Chinthagunta <xmlk...@gmail.com> > wrote: > >> I have same scheme in DB2 and SQL server

DB2 to MS SQL data transfer

I have same scheme in DB2 and SQL server. Is there a simple solution to transfer data with NiFi? ( one time ) Hoping for somebody have better idea to build the flow :) Thanks Sumo Sent from my iPhone

Re: DistributedMapCacheClient Groovy example

s the > capabilities as you have shown. Mind writing an improvement Jira to capture > this? > > Thanks, > Matt > > >> On Jul 17, 2016, at 4:51 AM, Sumanth Chinthagunta <xmlk...@gmail.com> wrote: >> >> I had to custom build NiFi to add distributed-cache

Re: ConsumeKafka vs getKafka

Done. https://issues.apache.org/jira/browse/NIFI-2298 <https://issues.apache.org/jira/browse/NIFI-2298> Thanks -Sumo > On Jul 17, 2016, at 1:37 PM, Sumanth Chinthagunta <xmlk...@gmail.com> wrote: > > Env: Linux , Kafka 9.0.x, NiFi: 0.7 > > We like to use

Re: DistributedMapCacheClient Groovy example

apr-build.md#add-optional> Please consider adding this support in next release. -Sumo > On Jul 16, 2016, at 10:17 PM, Sumanth Chinthagunta <xmlk...@gmail.com> wrote: > > Hi Matt, > > Did you find any solution to use DistributedMapCacheClientService from > Scripting pr

Re: DistributedMapCacheClient Groovy example

t; > I'll give it a try myself to see if I can find a way (using closures > -- but not interface coercion since I can't reference the class) to > achieve this, but otherwise you can use the first blog post from my > last email to write a simple client of your own inside Groovy. >

Re: DistributedMapCacheClient Groovy example

processor ! Thanks Sumo > On Jul 13, 2016, at 9:12 PM, Sumanth Chinthagunta <xmlk...@gmail.com> wrote: > > Matt, > I setup DistributedMapCacheServer controller service and trying to run > following script. Am I doing correct? > I have couple of is

Re: DistributedMapCacheClient Groovy example

method > invocation (as outlined in the blog post) you can call its methods to > get and put values. > > Regards, > Matt > > On Wed, Jul 13, 2016 at 8:26 PM, Sumanth Chinthagunta <xmlk...@gmail.com> > wrote: >> looking for example script ( for ExecuteScript processor) that uses >> DistributedMapCacheClient to put/get key/value pair. >> >> Thanks >> -Sumo

Re: DistributedMapCacheClient Groovy example

log post) you can call its methods to > get and put values. > > Regards, > Matt > >> On Wed, Jul 13, 2016 at 8:26 PM, Sumanth Chinthagunta <xmlk...@gmail.com> >> wrote: >> looking for example script ( for ExecuteScript processor) that uses >> DistributedMapCacheClient to put/get key/value pair. >> >> Thanks >> -Sumo

DistributedMapCacheClient Groovy example

looking for example script ( for ExecuteScript processor) that uses DistributedMapCacheClient to put/get key/value pair. Thanks -Sumo

Re: State sharing

he > default implementation backed by a ZK is not meant for high speed > concurrent use. Distributed caches are, however. > > Andrew > > On Sun, Jul 10, 2016, 10:58 PM Sumanth Chinthagunta <xmlk...@gmail.com> > wrote: > >> Thanks Bryan. >> Would be nice if we

Re: State sharing

, the state will be set to a ZK by default. Otherwise just >> local to this processor node. >> >> Andrew >> >> On Sun, Jul 10, 2016, 10:17 PM Sumanth Chinthagunta <xmlk...@gmail.com >> <javascript:;>> >> wrote: >> >>> If I se

Re: State sharing

gt; wrote: > > Sumo, > > IIRC there's a node one selects when setting state. If you invoke with a > cluster mode, the state will be set to a ZK by default. Otherwise just > local to this processor node. > > Andrew > > On Sun, Jul 10, 2016, 10:17 PM Sumanth Chinthagunta <

State sharing

If I set state from one ExecuteScript processor via stateManager , can I access same state from other processor ? Thanks Sumo Sent from my iPhone

Re: MergeContent to group related files

ess how to keep these > objects separate and avoid the need for header/foot/demarcator/etc.. Good > choice as well. > > There are a lot of ways to slice this up. > > Thanks > Joe > > On Wed, Jun 15, 2016 at 6:04 PM, Sumanth Chinthagunta <xmlk...@gmail.com &

Feature Request: NodeID of NiFi cluster nodes as FlowFile common attribute.

In my NiFi cluster environment, for a given flow, each node in the cluster is producing afinal FlowFile that is written to HDFS. I needed to append NodeID to filename attribute (e.g., node1_filename1.tar) so that files written to HDFS has unique name and I can easily identify which node

NiFi error with embedded ZooKeeper for State Management

I am getting following error when NiFi Cluster started with embedded ZooKeeper enabled. I would like to configure external ZooKeeper for NiFi State Management without interfering with java.security.auth.login.config setting for MapR HDFS. Can somebody provide me instructions how to

Re: [DISCUSS] Proposal for an Apache NiFi sub project - MiNiFi

Good idea. There will be many possibilities if we can make MiNIFi run on android / iOS or other embedded devices. Wonder how back-pressure works in this kind of distributed setup. I was reading about reactivesocket project, This project is trying to solve reactive / back-pressure problem over

Cluster Start Error: Apache NiFi is already running, listening to Bootstrap on port

Hi I am following clustering instructions as per the link below: NCM, Node1 in Server1 and Node2 on Server2. https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#clustering I created two

Re: Cluster Start Error: Apache NiFi is already running, listening to Bootstrap on port

multiple nodes executed concurrently > (like [S]FTP for instance). > > Matt > > On Fri, Dec 18, 2015 at 7:26 PM, Sumanth Chinthagunta <xmlk...@gmail.com > <mailto:xmlk...@gmail.com>> > wrote: > >> Issue 1: >> in my NiFi installation I granted on

Re: Cluster Start Error: Apache NiFi is already running, listening to Bootstrap on port

f7d-45d7-a1f4-f9e9269b489b/processors/bceff62d-435d-4c47-a907-bb2a26cd0e56]> while in safe mode please let me know if you find anything wrong with my setup. Thanks Sumo > On Dec 18, 2015, at 11:37 AM, Sumanth Chinthagunta <xmlk...@gmail.com> wrote: > > Thanks Corey and Ma

Re: [GitHub] nifi pull request: NIFI-1218 upgraded Kafka to 0.9.0.0 client API

Hi Oleg, For my case NiFi (consumer) has to talk to Kafka cluster behind firewall and it was difficult got Kafka brokers and ZooKeeper ports opened. I noticed that 0.9.0.0 support consumers directly communicate to Kafka brokers with `bootstrap.servers` setting. it might be good idea if you

Re: remote command execution via SSH?

e in Run DSL anything that is assigned to ‘result’ in RunDSL is passed as flowfile to success relationship. Any suggestions for improvements are welcome. -Sumo > On Nov 24, 2015, at 8:19 PM, Adam Taft <a...@adamtaft.com> wrote: > > Sumo, > > On Tue, Nov 24, 2015 at 10:

Re: remote command execution via SSH?

>> Oleg >> >>> On Nov 24, 2015, at 11:51 AM, Joe Witt <joe.w...@gmail.com> wrote: >>> >>> Hello Sumo, >>> >>> At present there are no such processors to do this. I know it has >>> been done in the past but that was not in

bundling shared jars

Sorry if this question may have been answered previously: I am implementing two custom processors (bundled separately as NAR files), both depends on Groovy Jar. instead of bundling Groovy Jars in each NAR, is there a way to share jars that don’t have any processors. Should we consider NiFi

remote command execution via SSH?

Are there any NiFi processors to execute remote commands via SSH? I need to SSH to a remoter server and run a shell script on schedule basses. thinking of using NiFi’s scheduling and argument passing capability. I find this lib can be used, if no such processor exist.

Re: NiFi won't start with my Processor

I noticed similar error when I have my flow using older version of a custom nar and when I deployed new version of nar with package refactoring. My workaround to bring the server back is to redeploy old nar, start nifi and delete the flow then recreate the flow after deploying refactored nar. Nifi

Re: NiFi won't start with my Processor

ke sure we understand what is happening here. > > Thanks > Joe > > On Wed, Oct 21, 2015 at 8:56 PM, Sumanth Chinthagunta <xmlk...@gmail.com> > wrote: > > I noticed similar error when I have my flow using older version of a > custom > > nar and when I deployed ne

MonitorActivity bug?

Noticed "copy attributes" is implemented only for REL_ACTIVITY_RESTORED case. I needed original attributes for REL_INACTIVE case as well. wonder why it is limited to activity restored relationship only.

Re: attributes to json

aceText to build a new Json document from flow file > attributes. Something like... > > Replacement Text = { "myField" : "${attribute1}" } > > -Bryan > > On Saturday, October 3, 2015, Sumanth Chinthagunta <xmlk...@gmail.com> > wrote: >

Re: need help with Async code

i/processors/standard/JmsConsumer.java > > <https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/JmsConsumer.java> > > On Sun, Sep 27, 2015 at 3:24 PM, Sumanth Chinthagunta <x

need help with Async code

Hi All, I am new to NiFi and I'm stuck with couple issues: 1. Unable to get hold of ControllerService from Processor’s init method. I wanted to pre-set some dependencies during init phase instead of querying them repeatedly in onTrigger method. I am getting null for

nifi question

Hi All, I am new to NiFi and I'm stuck with couple issues: 1. Unable to get hold of ControllerService from Processor’s init method. I wanted to pre-set some dependencies in my Processor during init phase instead of querying them repeatedly in onTrigger method. I am