Re: zookeeper error message - nifi 1.11.1/zookeeper 3.5.6

Dan, I believe the issue (if there is one) would be between Zookeeper and Curator, but NiFi uses Curator 4.x which is the correct version to use with ZK 3.5.x. On Thu, Feb 13, 2020 at 12:28 AM dan young wrote: > Thank you for your email. Looking at the zookeeper docs, with 3.5.0 it > looks

Re: Influence about removing RequiresInstanceClassLoading from AbstractHadoopProcessor processor

wrote: > Hi,jeff, > There is no Kerberos authentication in my HADOOP clusters , but I > find UGI is initialized with an ExtendConfiguration(extend from hadoop > Configruation) when those processor's instance was first scheduled .I would > use nifi to connect different

Re: Influence about removing RequiresInstanceClassLoading from AbstractHadoopProcessor processor

If you remove the @RequiresInstanceClassloading, the UserGroupInformation class from Hadoop (hadoop-common, if I remember correctly) will be shared across all instances that come from a particular NAR (such as PutHDFS, ListHDFS, FetchHDFS, etc, from nifi-hadoop-nar-x.y.z.nar). If you are using

Re: NiFi 1.10.0 and ZooKeeper 3.5.5

Pierre, thanks for confirming what I was seeing. Jeff On Fri, Nov 8, 2019 at 12:08 PM Pierre Villard wrote: > Hi Jeff, > > I came to the same observation on my side as well. Upgrading to ZK 3.5.5 > did solve my issues. > > Pierre > > Le ven. 8 nov. 2019 à 18:04,

NiFi 1.10.0 and ZooKeeper 3.5.5

this is correct and I'm not running into some other issue with NiFi 1.10.0 and ZooKeeper 3.4. I kind of expected the 3.5 client to be backward compatible with 3.4 but maybe it is not. Thanks, Jeff [1] https://issues.apache.org/jira/browse/NIFI-6578

Re: JAVA 11

Hello Clay, Currently you can run NiFi 1.9.2 on Java 11, though the binaries are built with Java 8. Java 11 build support is in the master branch, and is scheduled to be released in NiFi 1.10.0. Keep an eye out for the 1.10.0 release candidate, on which work is currently progressing. On Tue,

Re: Problem with Context Path Whitelisting

Swarup, Agreed with Kevin, very nice write-up on the scenario! Would you please provide the original request as sent by Nginx, along with your configuration pertaining to NiFi in Nginx? We can set up some test cases to reproduce what's happening and get a JIRA filed if there's an edge case not

Re: Nifi errors - FetchFile and UnpackContent

Hello Tomislav, Are these processors running in a multi-node cluster? Is FetchFile downstream from a ListFile processor that is scheduled to run on all nodes versus Primary Node only? Is FetchFile's Completion Strategy set to "Move File" or "Delete File"? Typically, source processors should be

Re: Question on MergeContent "Max bin age"

I like the idea of seeing the details of the reason for eviction/merge in the details of a provenance event. Those same details could be provided in an attribute as well. If a log statement was also created, it should probably be at the DEBUG level. On Fri, Aug 9, 2019 at 9:57 AM Mark Payne

Re: Kerberos Ticket Renewal (when not updating Hadoop user)

James, No worries! At least you now have another point of reference. :) On Thu, Jun 13, 2019 at 1:11 PM James Srinivasan wrote: > Err, my bad - meant to send this to the Accumulo list! > > Sorry! > > On Thu, 13 Jun 2019, 18:07 Jeff, wrote: > >> Hello James, >>

Re: Kerberos Ticket Renewal (when not updating Hadoop user)

Hello James, For our Hadoop processors, we generally don't do any explicit relogins/TGT renewal. It's handled implicitly by the Hadoop libs. PR 2360 [1] is the primary change-set to allow this in NiFi, and several NiFI JIRAs (mainly NIFI-3472 [2]) are referenced in that pull request if you are

Re: Number and size of nodes

Christian, Another factor to consider for the NiFi nodes is the disk hardware and configuration. Simply put, the faster your disks are (SSD, RAID0, etc), the faster NiFi will be able to perform. This is dependent on the needs of your flow; the number of CPUs/cores isn't the only consideration

Re: migrate state for local to cluster transition

Hello Ben, In NiFi Toolkit, the Zookeeper Migrator exists to move state from one ZK quorum to another, but to my knowledge, there's no local-to-cluster migrator. There is a JIRA [1] for this functionality, but it's currently unassigned, and I haven't heard about any work being done for it. [1]

Re: Apply Zookeeper ACL to Existing NiFi Cluster

t the flow to have it pick up where it left off, based on the imported state. You could do these for each NiFi cluster, providing a different root node for each cluster. Hopefully this helps, and again, sorry for the delay in response! Please let us know if you need more information. - Jeff On

Re: jdbc impala url

Hello, I'm working on some NiFi/Impala integration examples currently, and will be taking a look at kerberized Impala in a few days. I'll try to follow up with you on this list after I have things working. If you beat me to it, please let us know! - Jeff On Mon, Jan 7, 2019 at 8:32 PM PasLe

Add NiFi Registry client

missed it. Thanks, Jeff

Re: InvokeHTTP failure to schedule fro CRON?

Service that uses the custom truststore. This isn't a comprehensive guide on CA certs, but I hope it helps you to work through the issue. Please let us know if we can help further! - Jeff On Mon, Dec 24, 2018 at 9:34 AM l vic wrote: > Could you try using an explicit path to the cacerts prov

Re: InvokeHTTP failure to schedule fro CRON?

get a list of what is contained in cacerts by using keytool, and specifying the path to cacerts, the password, and the list command. For example: keytool -storepass changeit -keystore /Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre/lib/security/cacerts -list - Jeff On F

Re: Question regarding krb tgt renewal for Hive processors and connection pools

Pat, I apologize for not seeing this thread until today! I'm glad there was a commit available for you to cherry-pick and resolve your issue. Also, thanks to Shawn and Bryan the helpful comments! - Jeff On Sun, Dec 23, 2018 at 10:13 AM Pat White wrote: > Update, cherrypicking the fix f

HiveQL processors in Nifi 1.7.0

ate on the fly. Also, if anyone didn’t see any behavior problems when upgrading to Nifi 1.7.0 in AWS, I’d be really interested to hear which version of EMR/Hive you are using. Thank you for any suggestions, Jeff

Re: [EXT] ReplaceText cannot consume messages if Regex does not match

Is the actualSettlementDate attribute's value in the "MM/dd/" format, with no other text in front of the date? For instance, "10/18/2018 12:30:00" is parsable by the "MM/dd/" format, but "12:30:00 10/18/2018" is not. On Thu, Oct 18, 2018 at 12:02 PM Juan Pablo Gardella <

Re: question about ConsumeKafka metrics for incoming data

Dominique, The "bytes in" value represents how many bytes have been received from an incoming connection. Since ConsumeKafka is directly consuming data from Kafka (a "source" processor), and not from a connection from an upstream processor, it won't show a value for "bytes in". If you want to

Re: Whitelisting Proxy Host values in a Container Environment?

to resources hosted by NiFi are made through the proxy, instead of exposing the internal URI to a resource. - Jeff On Fri, Oct 12, 2018 at 12:42 PM Jon Logan wrote: > We are running into issues with NiFi not allowing secure connections in a > container due to the proxy...the only documen

Re: [EXT] Re: Hive w/ Kerberos Authentication starts failing after a week

Peter, They're in separate NARs, and are isolated by different ClassLoaders, so their state regarding UGI will be separate. There shouldn't be a problem there. The only way I could think of that might create a problem is if Atlas JARs were added to HDFS using the Additional Classpath Resources

Re: NiFi as a service

Tim, thanks a lot! I will give that a try. Mike, I used the "nifi.sh install" as Tim described. My workaround was just to remove /etc/init.d/nifi but your docker suggestion will help, too. Thanks! Jeff On Fri, Jun 15, 2018 at 1:55 PM Tim Dean wrote: > Mike - > > The sta

NiFi as a service

service and the MongoDB install finished ok. Wondering if there is anything I should do differently? If other information is needed please let me know. Thanks, Jeff

Re: Hive connection Pool error

Vishal Dutt, Your issue relates to an existing JIRA [1] and as luck would have it, it's already resolved! :) The fix for that JIRA [1] has been merged to master and will be in the next NiFi release. [1] https://issues.apache.org/jira/browse/NIFI-5134 [2] https://github.com/apache/nifi/pull/2667

Re: TLS Toolkit Certs: Knox to NiFi

Hi Ryan, I responded to your question over on the Knox user list, but I can include my response here as well. I'm glad you're using the TLS Toolkit, I was going to suggest you give that a try, initially. The cert from the keystore generated by the toolkit that identifies the cert to use for

Re: NiFi 1.5 with Knox 1.0.0

om)? I’m just wondering if accessing NiFi thru Knox >> will result in losing the concept of users. Hopefully this makes sense! >> >> Cheers, >> >> Ryan >> >> On Sun, Mar 4, 2018 at 1:33 PM Jeff <jtsw...@gmail.com> wrote: >> >>> Hello Ryan, >&

Re: NiFi 1.5 with Knox 1.0.0

Hello Ryan, I am not on my development laptop right now, but I can send you an example Knox topology that uses Knox, SSO, and NiFi. Regarding the two options you listed above, both can be used simultaneously. If you only want to use option 1, you can set the Knox properties in nifi.properties

Re: Large number of NiFi processors

Hello Jon, The number of processors is virtually unlimited, provided that you have enough CPU to sustain the number of concurrent tasks allocated to NiFi to provide acceptable performance, enough disk space for the NiFi repositories, and enough RAM to cover the overhead of the processor

Re: Generalizing a List / Fetch workflow

It might take a bit more than allowing input on that processor. One thing you might want to check is how the processor retains state for the path given in the configuration. If this path changes regularly due to EL or incoming flowfiles, the state management for the processor might not be

Re: MiNiFi contents-repository config problem

Sumanth, MiNiFi 0.3.0 added a no-op provenance repository that does not store any events. To use it specify the no-op provenance repository implementation like: Provenance Repository: provenance rollover time: 1 min implementation: org.apache.nifi.provenance.NoOpProvenanceRepository Jeff

Re: Nifi: how to split logs?

Sally, What are you trying to parse out of nifi-app.log? If you are looking for provenance information, it may be easier and more straightforward to query provenance directly. On Fri, Nov 24, 2017 at 6:13 AM sally wrote: > I want to use nifi logs ( i mean

Re: Wait for N Files before starting to process

Manish, You're welcome, and thanks to Koji for writing up that blog! On Mon, Nov 20, 2017 at 9:27 AM Manish Gupta 8 <mgupt...@sapient.com> wrote: > Thank Jeff. This is exactly what I was searching for. Thank You. > > > > > > *From:* Jeff [mailto:jtsw...@gmail.com] &g

Re: Wait for N Files before starting to process

Hello Manish, I was answering a dev-list question and provided a URL to Koji's blog about Wait/Notify processors [1]. Please take a look and feel free to ask questions about how you you can integrate the Wait/Notify processors into your flow. [1]

[ANNOUNCE] Apache NiFi 1.4.0 release.

Hello, The Apache NiFi team would like to announce the release of Apache NiFi 1.4.0. Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. Apache NiFi was made for dataflow. It supports highly configurable directed graphs of data routing, transformation,

Re: NiFi ram usage

averages around 250mb for me, a default bootstrap.conf. On Thu, Aug 31, 2017 at 7:59 AM Pierre Villard <pierre.villard...@gmail.com> wrote: > As Jeff, I'm a bit surprised by what you are experiencing. I've never > changed the default values of 512MB when working with NiFi on my laptop and

Re: NiFi ram usage

HI Adam, Can you provide some more detail about what your NiFi flow is like? Are you using custom processors? I regularly use NiFi with the default bootstrap settings without issue, but if you're bringing lots of data into memory, have lots of flowfiles being processed concurrently, etc, memory

Re: Nifi 1.4.0

I'd be happy to undertake RM duties for the 1.4.0 release. On Wed, Aug 16, 2017 at 3:31 PM BD International < b.deep.internatio...@gmail.com> wrote: > Hello, > > Just wondering if there was a release date planned for nifi 1.4.0? > > There are some really good features and fixs in it. > > Thanks

Re: publishmqtt with SSL

] org.apache.nifi.StdOut Valid from Thu May 11 20:25:52 CDT 2017 until Mon May 07 12:03:30 CDT 2018 https://www.dropbox.com/s/8ddzmezx0sporwa/nifi-bootstrap.log?dl=0 https://www.dropbox.com/s/609jgeqaxwnukun/nifi-app.log?dl=0 Thanks, Jeff Oxenberg From: Andy LoPresto <alopre...@apache.org<mailto:

RE: publishmqtt with SSL

Bumping this up as I’m still having an issue here; has anyone gotten publishmqtt working with SSL? Jeff Oxenberg From: Oxenberg, Jeff Sent: Tuesday, August 08, 2017 8:33 PM To: users@nifi.apache.org Subject: publishmqtt with SSL Hey, I’m trying to get NiFi to send mqtt messages to the Azure

Nifi controller passwords stored at rest

for your help. Jeff

publishmqtt with SSL

:498) Thanks, Jeff Oxenberg

Re: Using QueryDatabaseTable processor in MiNiFi

I went through the steps again and it worked fine so it was total user failure somewhere on my part. I am curious as to where I went wrong so I am trying to reproduce it to see if it's worth any documentation updates. Thanks, Jeff On Thu, Jul 27, 2017 at 10:09 AM, Aldrin Piri <aldr

Using QueryDatabaseTable processor in MiNiFi

INFO [main] o.a.nifi.groups.StandardProcessGroup DBCPConnectionPool[id=c42fe292-fe4b-3423--] added to StandardProcessGroup[identifier=f56cb9ff-f706-328b--] Did I forget to do something? Thanks, Jeff

Re: ListS3 duration

Laurens, I think I have a working Cloudtrail flow on my other computer... I'll try to fire that up today and see what I get. I used 1.3.0 the last time I looked at Cloudtrail data. On Thu, Jul 20, 2017 at 4:56 PM Laurens Vets wrote: > Please see inline for my answers and

Re: S3 Cloudtrail log ingestion to Kafka via NiFi

Hello Laurens, I set up a flow to test this as well, and also saw that the unzipped json looks like it contains references to other S3Objects. I'm not familiar with the formats used by CloudTrail or how the actual logging data is stored. I'll have to read up on it, but I think we can set up a

Re: Phantom node

Can you provide some information on the configuration (nifi.properties) of the nodes in your cluster? Can each node in your cluster ping all the other nodes? Are you running embedded ZooKeeper, or an external one? On Wed, May 3, 2017 at 8:11 PM Neil Derraugh <

Re: Organizing the flows in the web interface

There is no "New" button to create a blank canvas that would function as a separate flow. You can organize your flows into separate Process Groups, though. It's a visual distinction, and still subject to the policy hierarchy of Process Groups. Other parts of your flow can send and receive data

Re: How to identify MiNiFi source edge devices

in this thread. The Variable Registry along with the UpdateAttribute processor appears to satisfy that need cleaner than a custom processor. I hope that explains the situation a bit! Thanks, Jeff On Tue, Apr 25, 2017 at 11:17 AM, Aldrin Piri <aldrinp...@gmail.com> wrote: > Jeff, >

Re: How to identify MiNiFi source edge devices

It is possible. I will take a look to see if the hostname is sufficient for the device ID. I just learned about the Variable Registry. It seems if I use the Variable Registry to store the device ID it would be available to the UpdateAttribute processor. Is that correct? Thanks, Jeff On Tue

How to identify MiNiFi source edge devices

a processor that reads a configuration file on the edge device to get its device ID and put that ID as an attribute in the flowfile. This appears to work, but, I was wondering if there is a more recommended approach? Thanks, Jeff

Re: MiNiFi's differentiator

Thanks for that info, Aldrin! I will be glad to look over it. Jeff On Thu, Apr 20, 2017 at 12:09 PM, Aldrin Piri <aldrinp...@gmail.com> wrote: > Hey Jeff, > > Just to hone in a bit more, MiNiFi would likely consume from some of the > work surrounding the Command and C

Re: MiNiFi's differentiator

implementation. WholeConfigDifferentiator's implementation seems simple, fast, and to the point. I wasn't aware of the NiFi-Registry project so that helps me see the bigger picture. Thanks, Jeff On Thu, Apr 20, 2017 at 10:36 AM, Joe Percivall <jperciv...@apache.org> wrote: > Hello Jeff, > &g

Re: Configure ExecuteSQL in MiNiFi

Thank you, Aldrin! That is great to hear. Jeff On Thu, Apr 13, 2017 at 10:02 AM, Aldrin Piri <aldrinp...@gmail.com> wrote: > Hi Jeff, > > This support was provided and is slated for 0.2.0 via MINIFI-154 but has > not yet been released pending NiFi 1.2.0. From you

Configure ExecuteSQL in MiNiFi

How do I configure the database properties for the ExecuteSQL processor in MiNiFi? I created my flow in NiFi and used the tool to convert it for MiNiFi before I saw in the documentation that controller services are not supported by MiNiFi. Thanks for any pointers! Jeff

Re: How can datetime to month conversion failed in french language?

Month format. > > format('MM')-->It converts 'May' into Number like if Jan it should be 01 > then 'May it should be 05. > > Here i have convert month into number. > > Cheers > > On Mon, Apr 10, 2017 at 5:58 PM, Andre <andre-li...@fucs.org> wrote: > > Prab

Re: MaxFileSize in timeBasedFileNamingAndTriggeringPolicy not work?

t; > On Wed, Apr 12, 2017 at 3:58 PM, prabhu Mahendran <prabhuu161...@gmail.com > > wrote: > > Jeff, > > Thanks for your mail. > > I need to use log back 1.1.7 in NiFi-0.6.1 . > > Can you suggest any way to change 1.1.3 into 1.1.7? > > On Wed, Apr 12, 2017 at

Re: MaxFileSize in timeBasedFileNamingAndTriggeringPolicy not work?

Hello Prabhu, I think you're running into a logback bug [1] that is fixed with 1.1.7. Unfortunately, it looks like NiFi 0.6.1 is using logback 1.1.3. [1] https://jira.qos.ch/browse/LOGBACK-747 On Tue, Apr 11, 2017 at 8:37 AM prabhu Mahendran wrote: > In NiFi-0.6.1 ,i

Re: Is it possible to dynamically spawn a Processor Group ?

t for your processor, but flowfiles may be delayed a bit before they reach the downstream processors since a batch needs to be completed before the flowfiles are available to those processors. On Sat, Apr 8, 2017 at 5:08 AM Stephen-Talk <stephen.schem...@talktalk.net> wrote: > Jeff, > > Y

Re: Latency v Throughput

n be easily undone/redone within the > confines of NiFi and not have changed some external system state. So, > this is a really important thing to appreciate. > > Thanks > Joe > > On Fri, Apr 7, 2017 at 2:18 PM, Jeff <jtsw...@gmail.com> wrote: > > James, >

Re: Latency v Throughput

James, The way I look at it (abstractly speaking) is that the slider represents how long a processor will be able to use a thread to work on flowfiles (from its inbound queue, allowing onTrigger to run more times to generate more outbound flowfiles, etc). Moving that slider towards higher

Re: How can datetime to month conversion failed in french language?

Prabhu, I'll have to try this in NiFi myself. I'll let you know what I find. What is the result of the EL you're using when you are trying it with French? On Fri, Apr 7, 2017 at 1:03 AM prabhu Mahendran <prabhuu161...@gmail.com> wrote: > jeff, > > Thanks for your reply. &g

Re: Is it possible to dynamically spawn a Processor Group ?

that collects csv files > (>100,000 lines) which in turn queues the file and parses each line to a > locally built processor (MyImportProcessor say) that submits them via > the REST API to a Drupal website. > The process works fine, but it is very slow, and would like to speed it > up by split

Re: How can datetime to month conversion failed in french language?

What is the expression language statement that you're attempting to use? On Thu, Apr 6, 2017 at 3:12 AM prabhu Mahendran wrote: > In NiFi How JVM Check language of machine? > > is that take any default language like English(US) else System DateTime > Selected language?

Re: Is it possible to dynamically spawn a Processor Group ?

Hello Stephen, It's possible to watch the status of NiFi, and upon observing a particular status in which you're interested, you can use the REST API to create new processor groups. You'd also have to populate that processor group with processors and other components. Based on the scenario you

Re: What happens if Nifi JVM die? is there any way to monitor and restart it?

Hello, The NiFi System Administrator Guide [1] has a section on Bootstrap Properties [2], through which you can set up a Notification Service [3] for when NiFi dies. At this time, the only Notification Service provided by NiFi is an email notification service, as noted in the guide. The

Re: Wait for child file to process successfully

, 2016 at 3:33 AM BD International < b.deep.internatio...@gmail.com> wrote: > Jeff, > > Thanks for that just tried it out and it works perfectly! > > On a similar topic I have a flow which picks up a CSV and turns each row > into an AVRO object and posts that to a web serv

Re: Wait for child file to process successfully

ou'll have a recreation of the original zip file, and it signifies that all the child flowfiles have completed processing. - Jeff On Thu, Dec 22, 2016 at 12:29 PM BD International < b.deep.internatio...@gmail.com> wrote: > Hello, > > I've got a data flow which picks up a zip file and uses U

Re: Load-balancing web api in cluster

com> wrote: > Hi Jeff, > > I saw this and looked into it. The data in those nodes are > the nifi.cluster.node.address and nifi.cluster.node.protocol.port values. > In order to get the nifi.web.http.host and nifi.web.http.port values, it > seems I would have to connect first using t

Re: Load-balancing web api in cluster

Greg, That first statement in my previous email should read "which nodes can be the primary or cluster coordinator". I apologize for any confusion! - Jeff On Tue, Dec 20, 2016 at 2:04 PM Jeff <jtsw...@gmail.com> wrote: > Greg, > > NiFi does store which nodes are th

Re: Load-balancing web api in cluster

in ZK, just the nodes that are candidates to fulfill those roles. I'll have to get back to you on that for sure, though. - Jeff On Tue, Dec 20, 2016 at 1:45 PM Hart, Greg <greg.h...@thinkbiganalytics.com> wrote: > Hi Jeff, > > My application communicates with the NiFi REST API to i

Re: Load-balancing web api in cluster

Process Group to load balance connections to the REST API on each NiFi node and to balance the processing load across the cluster. [1] https://pierrevillard.com/2016/08/13/apache-nifi-1-0-0-cluster-setup/ - Jeff On Mon, Dec 19, 2016 at 9:25 PM Hart, Greg <greg.h...@thinkbiganalytics.com> wrote:

Re: merge flowfiles

Hello Raf, MergeContent can merge based on a correlation ID (attribute). However, the merging currently operates in two modes: Defragment or Bin-Packing Algorithm. Defragment is completed by defragmenting based on the correlation ID and a known number of fragments. Bin-Packing Algorithm is

Re: Data Provenance is not available

overridden for a child component, that user will be able to see data provenance for all components. On Mon, Nov 21, 2016 at 6:07 PM Jeff <jtsw...@gmail.com> wrote: > Hi Pablo > > From the global policies menu, you can add the users that need to able to > query Data Provena

Re: Data Provenance is not available

Hi Pablo >From the global policies menu, you can add the users that need to able to query Data Provenance to the "query provenance" policy. For any component, such as the root group, or a particular processor, you'll also want to add the user to "view the data" policies for the respective

Re: Nifi- PutEmail processor issue

Hello Sravani, Could it be possible that the SMTP server you're using is denying connections due to the volume of emails your flow might be sending? How many emails are sent per flow file, and how many emails do you estimate are sent per minute? If this is the case, you can modify your flow to

Re: How do you recover a workflow ?

flowfile clicking some > button instead to start/stop ad additional GenerateFlowFile processor. > > Perhaps I'm doing it in the wrong way. So how do you restart a workflow ? > Maybe this feature can help others in the same task. > Don't know... I'm just asking. > > > > >

Re: How do you recover a workflow ?

Hello Alessio, Could you provide some more details about your NiFi flow? One of the triggers I used to manually be able to start processing in my flow was to have a GetFile processor (configured with "Keep Source File" set to false) watching for files in a directory, and when I wanted to test

Re: Push x Pull ETL

Great to hear, Marcio! On Thu, Oct 13, 2016 at 9:26 PM Márcio Faria <faria.mar...@ymail.com> wrote: > Jeff, > > Many thanks. I'm now more confident NiFi could be a good fit for us. > > Marcio > > > On Wednesday, October 12, 2016 9:06 PM, Jeff <jtsw...@gma

Re: Push x Pull ETL

and restarting of processing when errors occur is possible, though much of that is in how you design your flow. Feel free to ask any questions! Much of the information above is fairly high-level, and NiFi offers a lot of processors to meet your data flow needs. - Jeff On Tue, Oct 11, 2016 at 5:18

Re: UI: feedback on the processor 'color' in NiFi 1.0

I was thinking, in addition to changing the color of the icon on the processor, that the color of the drop shadow could be changed as well. That would provide more contrast, but preserve readability, in my opinion. On Mon, Sep 19, 2016 at 6:39 PM Andrew Grande wrote: > Hi

Re: PermissionBasedStatusMergerSpec is failing

. Can you please change line 86 in FormatUtils to use Locale.getDefault() instead of Locale.US and rerun the tests? On Fri, Sep 16, 2016 at 1:12 PM Tijo Thomas <tijopara...@gmail.com> wrote: > Hi Jeff, > > Yes, I took a fresh clone from git hub . I cleaned my maven repo as well &g

Re: PermissionBasedStatusMergerSpec is failing

repo and > try again. Still the problem exist I will post it again in the group. > > Thank you very much for your support. > > Tijo > > On 10-Sep-2016 6:44 pm, "Jeff" <jtsw...@gmail.com> wrote: > >> Tijo, >> >> Have you modified ProcessorSt

Re: PermissionBasedStatusMergerSpec is failing

Tijo, Have you modified ProcessorStatusSnapshotDTO.java or PermissionBasedStatusMergerSpec.groovy? On Sat, Sep 10, 2016 at 7:48 AM Tijo Thomas <tijopara...@gmail.com> wrote: > Hi Jeff > > I recently rebase from master. > Then I cloned again and ran mvn package > > T

Re: PermissionBasedStatusMergerSpec is failing

Tijo, I just ran this test on master and it's passing for me. Can you provide some details about the branch you're on when running the tests? I see that tasksDuration is 00:30:00.000 when it's expecting 00:00:00.000, and that's why the JSON isn't matching. On Thu, Sep 8, 2016 at 4:58 PM Tijo

RE: Need to read a small local file into a flow file property

Yeah, that would work.. here’s a quick example in python that works on my local machine. https://gist.github.com/jeffoxenberg/327b0dfeaa6bb63882279dd290222582 Thanks, Jeff Oxenberg From: Andre [mailto:andre-li...@fucs.org] Sent: Thursday, August 25, 2016 8:41 AM To: users@nifi.apache.org

RE: Shell script execution through Nifi

it. Thanks, Jeff Oxenberg From: Gadiputi, Sravani [mailto:sravani.gadip...@capgemini.com] Sent: Monday, July 11, 2016 9:04 AM To: users@nifi.apache.org Subject: RE: Shell script execution through Nifi Hi Bryan, Thank you for solution. Here I am adding few more to my question. 1)Can we capture any

RE: PutCassandraQL failing on ISO-8601-formatted timestamp

04:36Z' Key: 'j.value' Value: '6.7' -- INSERT INTO test.test2 (sensor, ts, value) VALUES(?,?,?) Thanks, Jeff Oxenberg -Original Message- From: Matt Burgess [mailto:mattyb...@gmail.com] Sent: Tuesday, June 21, 2016 8:52 PM To:

PutCassandraQL failing on ISO-8601-formatted timestamp

Hey, As a learning exercise, I've created a flow that parses a kafka topic of json messages and inserts them into Cassandra. It's a three column table: id (text), ts (timestamp), and value (float). I set cql.args.x.type to the proper data types for each column. PutCassandraQL is failing

Re: Can't connect to Secure HBase cluster

Do you have a core-site.xml in your config? On Thu, Mar 31, 2016 at 4:27 AM, Guillaume Pool wrote: > Hi, > > > > I am trying to make a connection to a secured cluster that has phoenix > installed. > > > > I am running HDP 2.3.2 and NiFi 0.6.0 > > > > Getting the following

Re: splitText output appears to be getting dropped

Matt, Thanks a bunch! That did the trick. Is there a better way to handle this out of curiosity? Than writing out a single line into multiple files. Each file contains a single string that will be used to build a url. -Jeff On Thu, Feb 18, 2016 at 6:00 PM, Matthew Clarke <matt.cla

splitText output appears to be getting dropped

12.187 TypeDROP FlowFile Uuid7fa42367-490d-4b54-a32f-d062a885474a File Size14 bytes Component Id3b37a828-ba2c-4047-ba7a-578fd0684ce6 Component NamePutFile Component TypePutFile DetailsAuto-Terminated by failure Relationship Any ideas on what I need to change here? Thanks in advance, Jeff

Re: Version Control on NiFi flow.xml

simply store the gz file. In this case, do we have some tool to compare two flow.xml.gz for some subtle changes? I am sure the UI based auditing is helpful though. On Thu, Feb 18, 2016 at 11:07 AM, Joe Witt <joe.w...@gmail.com> wrote: > Jeff > > I think what you're doing is ju

Re: Version Control on NiFi flow.xml

a formatted XML to put it into our Git repository: cat conf/flow.xml.gz | gzip -dc | xmllint --format - On Thu, Feb 18, 2016 at 10:01 AM, Oleg Zhurakousky < ozhurakou...@hortonworks.com> wrote: > Jeff, what you are describing is in works and actively discussed > https://cwik

Version Control on NiFi flow.xml

Hi, As my NiFi data flow becomes more and more serious, I need to put on Version Control. Since flow.xml.gz is generated automatically and it is saved in a compressed file, I am wondering what would be the best practice regarding version control? Thanks, Jeff -- Data Bean - A Big Data Solution

Re: Generate URL based on different conditions

b.com/mattyb149/8fd87efa1338a70c > > On Tue, Feb 16, 2016 at 9:51 PM, Joe Witt <joe.w...@gmail.com> wrote: > >> Jeff, >> >> For each of the input files could it be that you would pull data from >> multiple URLs? >> >> Have you had a chance to le

Re: Does NiFi support Hot Deploy?

Thanks Joe. I thought NiFi Clusters, or Clustered NiFis are used for HA, Load Balancing, and Scalability, but it seems like it can use for modularization. On Tue, Feb 16, 2016 at 3:45 PM, Joe Witt <joe.w...@gmail.com> wrote: > Jeff > > Clustered NiFis (multiple nodes) is quite c

Re: Does NiFi support Hot Deploy?

, prefer simple flow.xml configuration and combine NiFi instances together for more complex scenarios? On Tue, Feb 16, 2016 at 3:16 PM, Joe Witt <joe.w...@gmail.com> wrote: > Jeff, > > Regarding hot deploy of new dataflows: > Absolutely. The ability to have interactive c

  1   2   >