I hadn't heard of that. For NiFi instances and clusters we have, we have
some custom reporting metrics and then some further custom API calls we
make to populate an ElasticSearch database and Kibana Dashboards.
I'd love to hear if this works out or if there are other ones out there.
Ryan
On
1.17.0 release (or perhaps a
> 1.16.3 if the current RC is not released).
>
> Regards,
> Matt
>
> [1] https://issues.apache.org/jira/browse/NIFI-10027
>
> On Mon, May 23, 2022 at 11:46 AM Ryan Hendrickson
> wrote:
> >
> > > "Note that as NiF
2022 at 2:16 AM Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com> wrote:
> https://issues.apache.org/jira/browse/NIFI-10027 Add image/webp support
> to the content viewer
>
> WebP is a modern image format that provides superior lossless and lossy
> compression for images
.
>
> Thanks
>
> On Thu, May 12, 2022 at 9:19 AM Ryan Hendrickson <
> ryan.andrew.hendrick...@gmail.com> wrote:
>
>> Hi all,
>> I'm curious if anyone has used the nifi-content-viewer to view images
>> with a mime.type of "image/webp".
>>
>> Whe
Hi all,
I'm curious if anyone has used the nifi-content-viewer to view images with
a mime.type of "image/webp".
When I load up an image with that mime.type it says "No viewer is
registered for this content type"
https://developers.google.com/speed/webp
Thanks,
Ryan
Joe - We're testing some scenarios. Andrew captured some confusing
behavior in the UI when enabling and disabling load balancing on a
relationship: "Update UI for Clustered Connections" --
https://issues.apache.org/jira/projects/NIFI/issues/NIFI-9236
Question - When a FlowFile is Load Balanced
ours.
> >
> > This will allow us to see the delta and if there appears to be any sort
> of leak. If you cannot share these then you can do that analysis and
> share the results.
> >
> > Nobody should have to restart nodes to keep things healthy.
> >
> > Joe
We have a daily cron job that restarts our nifi cluster to keep it in a
good state.
On Mon, Sep 6, 2021 at 6:11 PM Mike Thomsen wrote:
> > there is a ticket to overcome this (there is no ETA), but other details
> might shed light to a different root cause.
>
> Good to know I'm not crazy, and
, can you share the links?
>
> Thanks
> -Mark
>
> On Aug 16, 2021, at 9:32 PM, Ryan Hendrickson <
> ryan.andrew.hendrick...@gmail.com> wrote:
>
> Axel,
>We've had significant issues with 1.13.2 in a Cluster as well. We're
> working on a test config... Issues r
t cannot
> afford investing more time into this right now. But we'll certainly come
> back to this later. We have to...
>
> --- Ursprüngliche Nachricht ---
> Von: Ryan Hendrickson
> Datum: 17.08.2021 03:32:25
> An: users@nifi.apache.org
> Betreff: Re: Disfunctional cluster with v
Axel - this is the Jira Ticket for NiFi for the missing Sys Admin
Properties: https://issues.apache.org/jira/browse/NIFI-9029
On Mon, Aug 16, 2021 at 9:32 PM Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com> wrote:
> Axel,
>We've had significant issues with 1.13.2 in a Clu
Axel,
We've had significant issues with 1.13.2 in a Cluster as well. We're
working on a test config... Issues range from abandoned FlowFiles, single
Nodes locking the entire cluster, load balance relationships not working,
and undocumented nifi properties. We're reluctant to move to 1.14.0
exceeded
the object backpressure limit? Similar to how DistributeLoad uses "Next
Available"?
Thanks,
Ryan
On Mon, Aug 9, 2021 at 4:59 PM Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com> wrote:
> Hi all,
> To confirm, when using a NiFi Cluster, are the Relationship Sett
Hi all,
To confirm, when using a NiFi Cluster, are the Relationship Settings the
"Back Pressure Object Threshold" and "Size Threshold" per node node, or
cluster-wide?
For example, if we have a 10 node cluster and set the Back Pressure Object
Threshold to 100. Would we then expect the
, 2021 at 12:08 PM Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com> wrote:
> Elli replied to this, although it looks like his email got flagged for
> spam, so I'm replying with his comments to make sure it got through:
>
> We are experiencing this issue as well. We just
Elli replied to this, although it looks like his email got flagged for
spam, so I'm replying with his comments to make sure it got through:
We are experiencing this issue as well. We just upgraded from Nifi 1.11.4
to 1.13.2, and are running in to this issue where many of our high-usage
Nifi
Hi all,
Our team just upgraded to NiFi 1.13.2. We've noticed frozen
relationships where data just queues up until the Remote Process Group is
started/stopped again.
I don't have any specific stacktraces isolated yet, just curious if
anyone else has experienced any issues with communication
Hi NiFi Users,
I've got a scenario where we've set back-pressure through a set of
critical path processor relationships, which can "lock-up" the server from
ingressing additional data.
In the mini diagram below, Server 2 has back pressure fully. Once
Server 2 reaches its back-pressure
ot; is 64,814.
> 6. The lowest Reference is 1 (when it's not zero)
> 7. The highest Reference is 4,773
> 8. Some References have Swap Files (10,006) and others have FlowFiles (470)
> 9. There are 10,468 "In Use"
>
> Anything there stick out to anyone?
>
> Thanks,
&
e are 10,468 "In Use"
Anything there stick out to anyone?
Thanks,
Ryan
On Thu, Sep 17, 2020 at 2:29 PM Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com> wrote:
> Correction - it did work. I was expecting it to be in the same folder as
> where I ran nifi.sh from, vs NIFI_HOME.
&
Correction - it did work. I was expecting it to be in the same folder as
where I ran nifi.sh from, vs NIFI_HOME.
Reviewing it now...
Ryan
On Thu, Sep 17, 2020 at 1:51 PM Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com> wrote:
> Hey Mark,
> I should have mentioned the PutElast
s of
> the processor perform significantly better than 1 instance with 2
> concurrent tasks, that’s probably worth looking into.
>
> -Mark
>
>
> On Sep 17, 2020, at 11:38 AM, Ryan Hendrickson <
> ryan.andrew.hendrick...@gmail.com> wrote:
>
> @Joe I can't export th
gt;
> <https://www.linkedin.com/company/alert-logic>
>
>
>
>
>
> *From:* Joe Witt
> *Sent:* Thursday, September 17, 2020 10:19 AM
> *To:* users@nifi.apache.org
> *Subject:* Re: Content Claims Filling Disk - Best practice for small
> files?
>
>
>
> c
1.12.0
Thanks,
Ryan
On Thu, Sep 17, 2020 at 11:04 AM Joe Witt wrote:
> Ryan
>
> What version are you using? I do think we had an issue that kept items
> around longer than intended that has been addressed.
>
> Thanks
>
> On Thu, Sep 17, 2020 at 7:58 AM Ryan Hendrickson
Hello,
I've got ~15 million FlowFiles, each roughly 4KB, totally in about 55GB of
data on my canvas.
However, the content repository (on it's own partition) is completely full
with 350GB of data. I'm pretty certain the way Content Claims store the
data is responsible for this. In previous
help without trying it out to
> see.
>
> Hope this helps!
> -Mark
>
> https://issues.apache.org/jira/browse/NIFI-7812
>
>
>
> On Sep 15, 2020, at 5:42 PM, Ryan Hendrickson <
> ryan.andrew.hendrick...@gmail.com> wrote:
>
> Hi Mark,
>I'm using Next Avai
ter performance.
>
> Thanks
> -Mark
>
>
> > On Sep 15, 2020, at 4:41 PM, Ryan Hendrickson <
> ryan.andrew.hendrick...@gmail.com> wrote:
> >
> > Hello,
> >I've got 1 million plus FlowFiles (nothing I can do about the count),
> that goto a Distribu
Hello,
I've got 1 million plus FlowFiles (nothing I can do about the count),
that goto a DistributeLoad. The DistributeLoad with 2 threads, a run
duration of 1 sec can only sustain ~200,000 FlowFiles / five minutes.
Is there a better design pattern or a processor that takes a Batch Size
to
We keep our queue limit at 20,000 to keep data from swapping between
ArrayLists and Prioritized Queues. See bug:
https://issues.apache.org/jira/browse/NIFI-7583
You can also adjust that limit up in the nifi.properties.
On Sat, Sep 12, 2020 at 1:15 AM Chris Sampson
wrote:
> One thing we've not
Hi all,
I've got a NiFi running with a lot of small JSON files and I'm trying to
squeeze the most performance out of it.
I recently saw the new RocksDB FlowFile Repo (
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#rocksdb-flowfile-repository)
and was wondering what
ry Concat)
\
/
\---original---> *ReplaceText* (Prepend JSON Key)
---success--> /
Not the cleanest, but it gets it done.
Ryan
On Fri, Nov 30, 2018 at 3:43 PM Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com>
{uuid}") }
Is there an equivalent way to do that with LookupRecord and the
MongoDBLookupService?
Thanks,
Ryan
On Fri, Nov 30, 2018 at 11:19 AM Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com> wrote:
> My incoming FlowFIle is a valid JSON Object. The key names could be
> anythin
On Fri, Nov 30, 2018 at 10:54 AM Mike Thomsen
wrote:
> For the record side of things, you just need to create a schema that
> includes your existing JSON fields and a new branch that will have the
> enriched fields in it.
>
> On Fri, Nov 30, 2018 at 10:39 AM Ryan Hendrickson <
>
Thu, Nov 29, 2018 at 8:05 PM Otto Fowler
> wrote:
>
>> Sounds like you want to look at enrichment with the LookupRecord
>> processors and Mongo.
>>
>> https://community.hortonworks.com/articles/146198/data-flow-enrichment-with-nifi-part-3-lookuprecord.html
>>
Hi all,
I'm curious if the GetMongo processor can allow you can pass in a
FlowFile Document and enrich the original FlowFile document with the
results from the Query Result - or more generally store the result as a
NiFi Attribute, instead of replacing the content in the FlowFile.
I really
wrote:
>>
>>> Yes. Use a custom validator.
>>>
>>> On Tue, Aug 7, 2018 at 1:58 PM Ryan Hendrickson <
>>> ryan.andrew.hendrick...@gmail.com> wrote:
>>>
>>>> Yea, just finished testing, it works.. I added an additional property
>&g
validator?
Ryan
On Tue, Aug 7, 2018 at 1:53 PM Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com> wrote:
> Haha, yea I'm testing out some code-hacking here to see what I can do
> too. If it works, I can try to submit a Pull Request for it. I haven't
> done one before, and this
ike Jackson
>> is just turning that into an ISO8601 string without that level of
>> precision. A custom mapper for Date objects should be able to solve that.
>> I'll work it when I get some free time from the daily grind.
>>
>> Thanks,
>>
>> Mike
>>
Hi all,
I'm using GetMongo configured with JSON Type of "Standard JSON". The
document I've got in Mongo has a date field that looks like the following:
{
...
"date" : ISODate("2018-08-06T16:20:10.912Z"
...
}
When GetMongo spits it out, the date comes out as:
1] http://www.apache.org/legal/release-policy.html#what
> [2] http://www.apache.org/legal/release-policy.htm
>
> On Wed, Aug 1, 2018 at 12:36 PM Ryan Hendrickson <
> ryan.andrew.hendrick...@gmail.com> wrote:
>
>> Hi all,
>>I'd like to make the recommendation/disc
Hi all,
I'd like to make the recommendation/discussion topic that nightly builds
of the nar's files (and/or NiFi) are created. Sometimes I need a fix
sooner than the next 1.x version coming out. I don't mind building master
to get the latest, but it would be a lot more convenient to just
Mike,
Did you get a chance to check this out at all?
Thanks,
Ryan
On Mon, Jul 9, 2018 at 3:31 PM Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com> wrote:
> Mike,
>No, I'm just using the Mongo shell example as two things: (1) Something
> that can already be done
driver based on my reading of the
>>> javadocs. All of the update and replace functions require two separate
>>> documents to be passed to them. If you can point me to an official Mongo
>>> API that supports that, I'll be happy to take a look at it as a possible
>>
e
> feature for 1.8.
>
> Thanks,
>
> Mike
>
> On Mon, Jul 9, 2018 at 3:01 PM Ryan Hendrickson <
> ryan.andrew.hendrick...@gmail.com> wrote:
>
>> I'm not sure what the right answer here is to implement something,
>> especially thinking about the
flowfile which has the static content you’re looking for, and
> pass that to the PutMongo processor.
>
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Jun 27, 2018, at 7:
s.Bson-
>
> The single parameter update() method doesn't exist on MongoCollection.
>
> It can be pretty tricky going back and forth between the Mongo shell and
> the client APIs.
>
> On Wed, Jun 27, 2018 at 5:31 PM Ryan Hendrickson <
> ryan.andrew.hendrick...@gmail.com&
po, but it should be
> {"customId":{$customId}} followed by {$set: {"status":"Stage_2"}}. Your
> version was one big document. What we would expect is a query like {
> "customId": "XYZ"}. As mentioned, $customId is not a valid JSON value.
&g
Hi Mike,
Just curious - any other suggestions?
Thanks,
Ryan
On Thu, Jun 21, 2018 at 5:23 PM Ryan Hendrickson <
ryan.andrew.hendrick...@gmail.com> wrote:
> Thanks for the suggestions, can you clarify if you mean the NiFi Processor
> Property "Update Query", or the FlowFi
ocessors.mongodb.PutMongo/index.html
On Thu, Jun 21, 2018 at 4:57 PM Mike Thomsen wrote:
> Two things:
>
> 1. You need to use valid JSON. Your query is not a valid JSON example
> because some of the values are not quoted.
> 2. You need to make sure the update option is set to u
Hi,
I can't seem to figure out the right combo of parameters to get a
document to update in Mongo using the PutMongo processor and the $set
operator.
Try 1:
The incoming flowfile contains the customId: abc
NiFi Processor Properties:
Mode: update
Upsert: false
Update Query Key: No
50 matches
Mail list logo
