Re: 'On primary node' ListSFTP not working for new cluster

n the version - this is 0.60. > I tried an even simpler flow with just GenerateFlowFile and LogAttribute and > it does the same for me. > > > From: Mark Payne > To: users@nifi.apache.org; Tom Stewart > Sent: Wednesday, April 13, 2016 11:42 AM > Subject: Re: 'On prim

Re: 'On primary node' ListSFTP not working for new cluster

Tom, When you are running on Primary Node, do you see the "Tasks/Time" on the processor showing that tasks are running? If you right-click on the Processor and choose "View State", does it show anything in the table? Thanks -Mark > On Apr 13, 2016, at 11:59 AM, Tom Stewart wrote: > > I buil

Re: Controller service reference another service

Vincent, I moved users@nifi to the BCC and instead am redirecting this to the dev@nifi mailing list, as this is developer question moreso than a user question. Certainly, you can reference one controller service from another. Generally, controller services are referenced by using a PropertyDesc

Re: Help on creating that flow that requires processing attributes in a flow content but need to preserve the original flow content

Chris, Unfortunately, at this time, the EvaluateJsonPath requires that the JSON to evaluate be the content of the FlowFIle. There already does exist a ticket [1] that would allow you to specify an attribute to use as the JSON instead of requiring that it be the content only. Unfortunately, this

Re: ListHDFS cache (again)

Conrad, If you are using a version that used DistributedMapCacheClientService, it also was saving state in a local file, $NIFI_HOME/conf/state/ You would need to delete that file, as well. I am not sure though if it would notice that the file was deleted without restarting NiFi. That processor

Re: ListenSyslog & scheduling strategy

Conrad, The Processor will only be scheduled to run on the Primary Node. However, it will be initialized on all nodes. It looks like the initialization step is where the Processor is opening the listening port. We should probably reconsider this in the future. It may make sense to instead run

Re: NPE in ExtractText

Conrad, I have never seen this personally. However, given your description, I was able to create a unit test that replicates the issue. I created a JIRA for this [1]. It was easily replicated. We should have a fix available for the 0.6.0 release. Thanks! -Mark [1] https://issues.apache.org/jir

Re: NPE in ExtractText

Conrad, What version of NiFi are you running? There was a bug that was introduced to the latest 'master' branch a few days ago that caused this, but it has since been fixed. If you are using an already released version, then we will obviously need to dig a little deeper. Thanks -Mark > On Mar

Re: Search provenance data based on an attribute

Sudeep, You are able to configure which attributes you want to have indexed by updating the "nifi.provenance.repository.indexed.attributes" property in conf/nifi.properties. You can add whatever attributes you'd like there (a comma-separated list). However, once you update that and restart NiFi,

Re: A couple of general questions

Chris, Welcome to the NiFi community! Regarding scaling out: The same flow runs on all nodes in a cluster. So all nodes run all Processors. The only exception to this is if you use the Scheduling Strategy of "Primary Node" - in that case, the Processor will run on only a single node. The idea h

Re: Processor Isolation Problem

Paul, You mentioned that PutFile is running on Primary Node only. Is the FlowFile queued on one of the other nodes? You can generally check this by right-clicking on the connection and clicking "List Queue" (assuming that you're running 0.5.x). Generally, you would run a Processor as Primary Nod

Re: UI performance slow

Conrad, Just to provide a bit of clarification, when Matt says "You can enable additional logging on the NCM", this is done by adding the following to your $NIFI_HOME/conf/logback.xml on the NCM: Thanks -Mark > On Mar 3, 2016, at 1:26 PM, Matt Gilman wrote: > > Conrad, > > That spinner ic

Re: Connecting Spark to Nifi 0.4.0

Kyle, Another thought here, that I don't see mentioned yet is how NiFi's site-to-site works. Because the Spark Receiver does not know whether the NiFi you're talking to is a standalone instance or a cluster, it first will ask for a list of all NiFi nodes. Assuming that you've got a standalone i

Re: Error starting Nifi Locally

Naveen, Do you have an instance of NiFi already running? You should be able to determine this easily by running "ps -ef | grep nifi" (assuming you're running on Linux). If so, you may need to shut down the current instance by doing a "kill -9". Thanks -Mark > On Feb 2, 2016, at 4:58 PM, Madhi

Re: ListS3 processor?

Adam, Just read through your post - fantastic write-up! Just wanted to say thanks for sharing. This is a question we've seen a few times in the last couple of weeks, and this is a great resource to point people to. Thanks -Mark > On Jan 31, 2016, at 1:57 AM, Adam Lamar wrote: > > Kyle/Joe, >

Re: Processor Signaling

Frank, The approach that you laid out here would certainly work. However, I feel that it may be a bit heavy-handed. If the idea is just to reload some information from a local file, then I would recommend that you take a look at the ScanContent / ScanAttribute Processors. They do something simil

Re: nifi fail to start up

rw-r--r-- 1 root root 5890046 Jan 15 17:19 2328.prov.gz > drwxr-xr-x 2 root root4096 Jan 15 17:22 index-1452874448389 > -rw-r--r-- 1 root root 1184678 Jan 15 17:22 23829647.prov.gz > drwxr-xr-x 2 root root 12288 Jan 15 17:22 toc > drwxr-xr-x 2 root root4096 Jan 15 17:31

Re: nifi fail to start up

Chakri, Can you send the results of performing an: ls -ltr on that directory? > On Jan 15, 2016, at 12:56 PM, Chakrader Dewaragatla > wrote: > > Mark - I see quite a few files and all has latest mtime . > > Thanks, > Chakri > > Sent from my iPhone > >

Re: nifi fail to start up

Chakri, Thanks for the stack trace. I have created a ticket to address this: https://issues.apache.org/jira/browse/NIFI-1404 In the meantime, it appears that the bug is due to the fact that you have a Provenance Event file that has no events, a

Re: nifi fail to start up

Chakri, Is there more to the stack trace? It seems like this is a partial stack trace. I would expect at least another "Caused by" clause. Thanks -Mark > On Jan 15, 2016, at 12:34 PM, Chakrader Dewaragatla > wrote: > > Hi – Nifi ran out memory and shutdown the service. When I try to start,

Re: Inserting data into mysql

John, You can use the ReplaceText processor to change the content of the FlowFile into whatever text you want it to be. You can use the Replacement Strategy of "Always Replace" and set the Replacement Value to the SQL that you want to use. Then you can send this to PutSQL. Does this give you w

Re: PutSQL question

Ralph, Hello. An easy way to do this is to use ReplaceText to set the content of the FlowFile. You can set the Replacement Strategy to "Always Replace" and set the Replacement Value to something like: INSERT INTO MY_TABLE (ID) VALUES ('${id}') You could also use SQL parameters (though it's slig

Re: Nifi cluster features - Questions

existing) feature. > - In a nifi cluster setup, can we group the machines and set site-to-site to > individual group. > For instance I have 10 node cluster, can I group them into 5 groups with two > nodes each. Run processors on dedicated group (using site to site or ot

Re: Nifi cluster features - Questions

ndpoints, > >> you could have the first relationship from DistributeLoad going to a > >> PostHTTP that posts to endpoint #1, and the second relationship going to a > >> second PostHTTP that goes to endpoint #2. > >> > >> If you want to distribute the data

Re: Testing a nifi flow via junit

Vincent, I would be vary wary about trusting performance results that you obtain by using the Mock Framework. The mock framework is intended to be used only for testing correctness, not performance. It has very different threading characteristics than the "actual" NiFi framework, and it uses ve

Re: The EndPoints of GetTwitter

We use the Hosebird client provided by Twitter. It does expose a Firehose endpoint but of course your tokens would have to be validate for that endpoint. I've never actually tested it, since I don't have credentials for the Firehouse, but I would love to hear the results of using it. I believe i

Re: MergeContent/SplitText - Performance against large CSV files

s://issues.apache.org/jira/browse/NIFI-1161 > <https://issues.apache.org/jira/browse/NIFI-1161> > > -Bryan > > On Mon, Nov 23, 2015 at 4:58 PM, Mark Petronic <mailto:markpetro...@gmail.com>> wrote: > Mark Payne stated, "We should have a RouteC

Re: Content replacement

Chandru,This ticket has now been merged to master. This feature will be available in 0.4.0, which should be out next week. If youprefer to build from source, it is now available for use from master. You should be able to now configure ReplaceText toeasily replace the values:Here, we use the Search

Re: Multiple Content Repositories

me curious when you mentioned that one or more FlowFiles can be > > written to the same Resource Claim. Is there a specific scenario wherein > > this can occur? Under normal circumstances there is only one FlowFile > > written to a Resource Claim? > > > > -- > > Chris > > &g

Re: Recovering from heap memory exceeded

Chris, Before you restart NiFi, if you look in the conf/nifi.properties file, there is a property called "nifi.flowcontroller.autoResumeState". If this is set to false, then on restart, the processors will be stopped instead of running. This will keep you from getting back into the OutOfMemoryE

Re: UDP Site-to-Site

> each node. > > Could a similar mechanism be implemented easily using NiFi? > > -Francois > > -Original Message- > From: Mark Payne [mailto:marka...@hotmail.com] > Sent: Friday, December 04, 2015 10:20 > To: users@nifi.apache.org > Subject: Re: UDP Si

Re: UDP Site-to-Site

Francois, Currently, the site-to-site protocol uses only TCP. Can you explain the use case that you had in mind so that we can understand how we might be able to improve this going forward, or perhaps provide suggestions of how else to accomplish your goals? Thanks -Mark > On Dec 4, 2015, at

Re: DistCp from Amazon S3 to HDFS

We certainly can do the reverse case - sync S3 with HDFS. With S3, as Joe S mentioned, we really should have a ListS3 but currently do not (We do have a ListHDFS though). Typically the use case that I've used with S3 is to setup S3 to notify when an object arrives via SQS. Then have GetSQS get th

Re: archive files

bin is being merged. Thanks -Mark [1] https://issues.apache.org/jira/browse/NIFI-1232 <https://issues.apache.org/jira/browse/NIFI-1232> > On Nov 30, 2015, at 3:11 PM, Mark Payne wrote: > > Charlie, > > One thing that you should note, specifically when using the Correl

Re: archive files

Charlie, One thing that you should note, specifically when using the Correlation Attribute is the property. If the value that you are using for the Correlation Attribute varies quite a bit, you could quickly fill up the default number of bins (100). In this case, it won't be able to add a FlowF

Re: Concept of async event-driven processors but not the experimental scheduling policy version

Mark, I can't say that I've ever really given thought to an explicitly "Eventing Model" like the one that you are describing. However, the way that you are describing it is really just a new relationship on the PutHDFS processor, so it would be a very processor-specific change. Rather than addi

Re: Multiple Content Repositories

Chris, In terms of round robin-ing between the repositories, yes, it follows a simple round-robin approach. In terms of sections within those containers, the answer is more of a "sort-of." Each FlowFile has what we refer to as a Resource Claim, which points to a location in the content reposito

Re: ExecuteProcess processor cannot be stopped/started

Chris, Just to expand upon what Joe mentioned below. When you say that it immediately becomes stuck when you restart, I assume that means that you're restarting with it running. If you tell the processor to stop and restart, it should not begin running. This way you wouldn't need to change the a

Re: dual nics

Pompilio, Hello. You should be able to achieve this by using the "nifi.web.http.host" and "nifi.web.https.host" properties in the conf/nifi.properties file. If unspecified, the default is to bind to all IP addresses, so by default you should be listening on both NIC's. Is this not the behavior

Re: TailFile: Wildcards and filenames

Andre, I love that you're digging in here and making sure that the quality is here! Thank you. So in your example here, you said that you have Rolling Filename Pattern set to: * So that should match any file in the directory (except that it won't count the actual file being tailed). From the

Re: Replicate flow files to multiple processors

Correct, the content/data itself is not copied. A replica of the FlowFile object is created, pointing to the same piece of content in the Content Repository. > On Nov 14, 2015, at 11:00 AM, Mark Petronic wrote: > > Mark, just wanted to clarify for my understanding. You said... > > "NiFi does

Re: MergeContent/SplitText - Performance against large CSV files

r example, imagine that we want to have a RouteCSV Processor such that it > is configured with multiple Regular Expressions." > > LOL. Guess someone had the baby in mind for awhile, too. :) > > On Fri, Nov 13, 2015 at 12:57 PM, Mark Petronic <mailto:markpetro...@gmail.com&

Re: MergeContent/SplitText - Performance against large CSV files

Is RouteText available on any branch where I could build and play around with > it before 0.4.0? > > Thanks, > Mark > >> On Fri, Nov 13, 2015 at 11:14 AM, Mark Payne wrote: >> Mark, >> >> That's a great question! 200 million per day equates to about 2K

Re: MergeContent/SplitText - Performance against large CSV files

Mark, That's a great question! 200 million per day equates to about 2K - 3K per second. So that is quite reasonable. You are very correct, though, that splitting that CSV into tons of one-line FlowFiles does indeed have a cost. Specifically, the big cost is the Provenance data that is generated

Re: High CPU usage in FileSystemRepository.java

Adam, What version of Java are you running? Do you have the following lines from conf/bootstrap.conf uncommented, or are they still commented out? java.arg.7=-XX:ReservedCodeCacheSize=256m java.arg.8=-XX:CodeCacheFlushingMinimumFreeSpace=10m java.arg.9=-XX:+UseCodeCacheFlushing java.arg.11=-XX:

Re: Queue wasn't emptying

wrote: > To answer Christopher, > You can clear out flowfiles on a queue by setting expiration on relationship > line. > > > On Thu, Nov 12, 2015 at 7:53 AM, Christopher Hamm <mailto:ceham...@gmail.com>> wrote: > I will look, but how do I just kill it with

Re: Queue wasn't emptying

2015, at 7:53 AM, Christopher Hamm wrote: > > I will look, but how do I just kill it with fire. What can I delete? > > On Nov 12, 2015 7:42 AM, "Mark Payne" <mailto:marka...@hotmail.com>> wrote: > Chris / Elli, > > What version of NiFi are you running?

Re: Why does PutFile create directories for you but PutHDFS does not?

Mark, My guess is that it was an oversight. I don't believe it was intentional to leave it out of PutHDFS. Thanks -Mark > On Nov 12, 2015, at 1:57 AM, Mark Petronic wrote: > > Just wondering about the history behind why one has the logic to > create them but the other does not?

Re: Queue wasn't emptying

Chris / Elli, What version of NiFi are you running? I am wondering if you may have been bitten by this bug [1]. This should affect only the 'master' branch, not any formally released versions of NiFi. If you built the application from source code, rather than using an already-released version,

Re: Replicate flow files to multiple processors

; mailto:users@nifi.apache.org>> >> Subject: RE: Replicate flow files to multiple processors >> >> Thanks Mark. This should help. >> >> Our use case is to route traffic (flowflies) to multiple independent >> processors that inline route to their respective workflows

Re: Memory Issues on Split Text

Naveen, There is a ticket [1] that will make this work more cleanly so that we can use SplitText to split a large file into millions of FlowFiles. Right now, as you noted you will end up running out of memory. There are a few possible solutions that you can use. If you need to split each line i

Re: Replicate flow files to multiple processors

Chakri, This can be done with any Processor. You can simply drag multiple connections that have the same Relationship. For example, you can create a GetSFTP processor and draw a connection from GetSFTP to UpdateAttribute with the 'success' relationship. and then also draw a connection from GetS

Re: adding new property in getkafka processor

Naveen, Fantastic! Would be happy to help if you run into any issues. Thanks! -Mark > On Nov 10, 2015, at 2:10 PM, Madhire, Naveen > wrote: > > I would like to try this one. Let me see if I can make the changes and let > you know if there are any issues. > > Thanks >

Re: adding new property in getkafka processor

Naveen, This is certainly something that should be configurable. Currently, it is always using the value of "smallest". I have created a ticket to make this configurable in the Processor: https://issues.apache.org/jira/browse/NIFI-1146 Thanks

Re: Help me figure out why this wont work

Christopher, You can look into the EvaluateXQuery, EvaluateXPath, and TransformXml processors to extract data from XML files. You can then use ReplaceText to build the SQL statement that you want to use and PutSQL to update the database. Thanks -Mark > On Nov 9, 2015, at 3:42 PM, Christopher

Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue

Joe, FWIW, I would personally avoid adding a Regex property. I think we use regexes already too much. I would just add the HTTP Headers or not... or you could even add a property that specifies whether or not to add the headers as attributes. This is very much a preference here, though. I think

Re: nifi stale flow file

ifi.apache.org>" > mailto:users@nifi.apache.org>> > Subject: Re: nifi stale flow file > > Mark Thanks, I moved to another directory to keep it as a backup. > I moved it back to flow.tar and cluster is back online. Make sense, I will > make sure workflows are sto

Re: nifi stale flow file

:53 PM, Chakrader Dewaragatla > wrote: > > Mark Thanks, I moved to another directory to keep it as a backup. > I moved it back to flow.tar and cluster is back online. Make sense, I will > make sure workflows are stopped prior restart. > > Thanks, > -Chakri > &g

Re: nifi stale flow file

Chakri, When the Cluster Manager sends a request to nodes and that request will change the flow, it marks its flow as "stale" until it hears back from the nodes whether or not the request was successful. So it appears that you restarted while a request was being replicated to the nodes. Typical

Re: GetSQS causes high CPU usage

Adam, I wonder if this ticket that I just created [1] is actually the same issue that you're seeing here. When GetSQS determines there is nothing to do, it will "yield", essentially pausing itself for some amount of time (by default 1 second). But the framework wasn't properly pausing the proce

Re: Client Site to Site

l need truststore property? > > > nifi.security.truststore= > nifi.security.truststoreType= > nifi.security.truststorePasswd= > nifi.security.needClientAuth=false > > > > > > > From: Mark Payne mailto:marka...@hotmail.com>> > Reply-To: "users@

Re: Client Site to Site

Naveen, With the config provided below, you are setting the "nifi.remote.input.secure" flag to true. This means that you will need to also set the keystore and truststore properties. Thanks -Mark > On Nov 3, 2015, at 12:36 PM, Madhire, Naveen > wrote: > > Hi, > > I am unable to connect to

Re: Is it just me or do others think it should be a bit harder to delete your components?

Matt, Just as an FYI, this was changed recently (perhaps 0.3.0?) so that you can delete a process group that is not empty. However, it does have to have all components stopped, no incoming connections, and all connections must be empty. Thanks -Mark > On Oct 31, 2015, at 3:05 AM, Matthew Cla

Re: User interface

Charlie, If you select one component on the graph, you can hold Shift and click other components to select multiple. You can also hold shift while dragging on the canvas in order to draw a bounding box around the components that you want to select. Ctrl+A will select all components. You can th

Re: Determine which Nifi processor eating CPU usage

Elli, Unfortunately, Java doesn't provide fine-grained details about which threads are using how much CPU, etc. However, the way that we typically tackle this is to go to the Summary view in NiFi (the table that you are talking about here). There are a couple of points of interest here. If you s

Re: ExecuteStreamCommand processor for "tail -n +2" not working as expected

Joe, Ultimately, we couldn't change the behavior without breaking backward compatibility. We do have a ticket [1] to add an "Argument Delimiter" property that is completed and will be included in 0.4.0. It will default to semi-colon in order to maintain backward compatibility but it can be cha

Re: Provenance doesn't work with FetchS3Object

Ben, Since FetchS3Object is not creating the FlowFile, it should not be emitting a RECEIVE event. This is certainly a bug. I have created a ticket for this: https://issues.apache.org/jira/browse/NIFI-1038 Thanks -Mark > On Oct 15, 2015, at 3

Re: Nifi Clustering - work distribution on workers

Mans, Nodes in a cluster work independently from one another and do not know about each other. That is accurate. Each node in a cluster runs the same flow. Typically, if you want to pull from HDFS and partition that data across the cluster, you would run ListHDFS on the Primary Node only, and th

Re: Problem with HandleHttpRequest/HandleHttpResponse

Seb, Thanks for all of the information. The stack trace certainly is helpful. This appears to be a bug. I have created a ticket [1] for this issue. The FlowFile certainly should be routed to failure. The ticket is currently assigned to the 0.4.0 version. Thanks -Mark [1] https://issues.apache

Re: Nifi & Spark receiver performance configuration

Aurelien, The way that swapping works in NiFi is that when the number of FlowFiles in a particular queue builds up past a certain point, NiFi will write those files to disk and drop them from the Java heap in order to avoid running out of heap space. Then, when the number of FlowFiles in the que

Re: Nifi 0.3.0 on java-1.8.0 (S3 processor error)

: "users@nifi.apache.org <mailto:users@nifi.apache.org>" > mailto:users@nifi.apache.org>> > Subject: Re: Nifi 0.3.0 on java-1.8.0 (S3 processor error) > > Mark – Yes, I have the right date/time with ntp client configured. > > Thanks, > -Chakri > > From: Ma

Re: Nifi 0.3.0 on java-1.8.0 (S3 processor error)

Chakri, Can you check that your system's date/time is accurate? I have not tried this with S3, but I know that Twitter sends back a pretty similar response when we use GetTwitter on a node that has the wrong date/time. It has to do with the authentication protocol that is used requiring an accu

Re: Nifi cluster features - Questions

ListenHTTP processor is running as a dafult (with out explicit setting to > run on primary node), how does the data transferred to rest of the nodes? > Does site-to-site come in play when I make one processor to run on primary > node ? > > Thanks, > -Chakri > > Fr

Re: Nifi cluster features - Questions

Hello Chakro, When you create a cluster of NiFi instances, each node in the cluster is acting independently and in exactly the same way. I.e., if you have 5 nodes, all 5 nodes will run exactly the same flow. However, they will be pulling in different data and therefore operating on different dat

Re: removing adding node to cluster

> Removing flow.xml.gz and all the template files worked. > > Thanks for your help >> On Sep 24, 2015, at 4:19 PM, Mark Payne wrote: >> >> Jeff, >> >> In this respect, it is also considering your templates part of your flow. So >> there's a good chance y

Re: removing adding node to cluster

Jeff, In this respect, it is also considering your templates part of your flow. So there's a good chance you've got a conflicting set of templates. If you blow away your flow.xml.gz and all templates you should be okay. Templates are stored in conf/templates, if you haven't changed the location

Re: GetHTTP Usage with Dynamic Filenames

Indus, As Daryl mentioned, InvokeHTTP will allow you to use the Expression Language to do an HTTP GET. It works a bit differently, though, because GetHTTP is a "Source Processor" whereas InvokeHTTP needs to be fed a FlowFile to do anything. So you can use GenerateFlowFile as a source and have i

Re: Generate flowfiles from flowfile content

One thing to note, if trying to pull a file from the local file system is that there is a ticket already [1] that would allow us to pull a file from the local file system using an attribute value. I know this ticket is actively being worked, but I don't know exactly when we are expecting to have

Re: ExecuteSQL Processor / ArrayIndexOutOfBounds Exception

Toivo, I think for VARBINARY, LONGVARBINARY, ARRAY, BLOB, and CLOB, I would use an Avro Bytes type. Basically, anything that could be serialized into a byte array i'd use a bytes type. Otherwise, I'd probably just either fail or skip that column. Not sure what else we can really do with this,

Re: How to recover flow graphics after running NiFi.sh install?

the flows > when I first ran as root. > > On Fri, 18 Sep 2015 at 11:10 pm Mark Payne <mailto:marka...@hotmail.com>> wrote: > Chris, > > We do have some work to do here to make the run.as <http://run.as/> work > better. Currently, the way that it works >

Re: How to recover flow graphics after running NiFi.sh install?

Chris, We do have some work to do here to make the run.as work better. Currently, the way that it works is that you will see the bootstrap run as root, and it launches a new process using "sudo -u ..." That process that is launched with "sudo -u" is the NiFi instance, so that it should be run a

Re: How to recover flow graphics after running NiFi.sh install?

Chris, I want to make sure that we all are on the same page here about what is going on. I'm wondering if the flow.xml.gz got destroyed somehow when switching between users or if NiFi was just unable to read it due to permissions and what not. Can you check the size of the conf/flow.xml.gz fil

Re: Help with ExecuteSQL

arseException for Avro telling me > that there's an empty name. I've tried several different ways of aliasing the > sql columns but I'm still getting the error. Ideas? > > On Mon, Sep 14, 2015 at 6:57 AM Mark Payne <mailto:marka...@hotmail.com>> wrote: > Tyle

RE: Help with ExecuteSQL

Tyler, The ExecuteSQL Processor expects to have incoming connections and will only run if it has an incoming FlowFile. I.e., it cannot be used as a "Source Processor." There is a ticket [1] to allow it to be the first Processor in the flow, but right now it will not run unless it has input. On

RE: ExtractText usage

imple" data to an attribute. There are some important > distinctions between this functionality and those in the ticket but the > core need is the same. May not smash into one processor but if this > generates a ticket please link to consider both in the process. > On Tue, Se

RE: ExtractText usage

isn't a simple way to > promote "simple" data to an attribute. There are some important > distinctions between this functionality and those in the ticket but the > core need is the same. May not smash into one processor but if this > generates a ticket please link to

RE: ExtractText usage

3 PM, Christopher Wilson > mailto:wilson...@gmail.com>> wrote: > I've moved the ball a bit closer to the goal - I enabled DOTALL Mode > and increased the Capture Group Length to 4096. That grabs everything > from the first line beginning with "R" to some of the &

RE: ExtractText usage

Agreed. Bryan's suggestion will give you the ability to match each line against the regex, rather than trying to match the entire file. It would result in a new FlowFile for each line of text, though, as he said. But if you need to rebuild a single file, those could potentially be merged togethe

RE: NiFi-Spark receiver not serializable

t; one. > > Thanks again, > Michael > > -Original Message- > From: Mark Payne [mailto:marka...@hotmail.com] > Sent: 04 September 2015 17:24 > To: users@nifi.apache.org > Subject: RE: NiFi-Spark receiver not serializable > > Michael, > > I have submitted a pat

RE: [External] Re: GitHTTP out of memory error

Mattie, Sounds good. If you increase the heap (I'd recommend 2 or 4 GB) and you are still running into OutOfMemoryError, then it would certainly be hinting at a memory leak of some sort. Please let us know if increasing the heap size helps! Thanks -Mark > Fro

RE: GitHTTP out of memory error

Mattie, How large is the Java heap? This value is in the conf/bootstrap.conf file. You'll typically see something like: java.arg.3=-Xmx512m Do you have any custom processors running in your flow, or are you using only Apache NiFi provided processors? Thanks -Mark

RE: Kafka+Avro processor chain

Hello Srikanth, The GetKafka Processor should be retrieving the value of a message from Kafka as a sequence of bytes and adding whatever it receives as the content of the FlowFile. It does not attempt to interpret it as a String. The key of the message, if any, is added as a FlowFile attribute,

RE: NiFi Inspector

Hi Rick, I believe that the Interactive Queue Management feature proposal does in fact encompass the feature that you are requesting. It does work slightly differently, though. Whereas you are suggesting that you want to add a processor to the graph, the Queue Management feature would mean that

RE: InvokeHTTP processor loses SSL Context after NiFi restart

Wow, let me try that again. John, I created the following ticket: https://issues.apache.org/jira/browse/NIFI-825. I was able to get the same error message using InvokeHTTP to do a GET against NiFi itself. Interestingly, it always would fail for me, though, not just after a restart. I updated

RE: InvokeHTTP processor loses SSL Context after NiFi restart

John,I created the following ticket: https://issues.apache.org/jira/browse/NIFI-825"; target="_blank" style="font-size: 12pt;">https://issues.apache.org/jira/browse/NIFI-825I was able to get the same error message using InvokeHTTP to do a GET against NiFi itself. Interestingly, it always would

<    2   3   4   5   6   7