I have a groovy scripts that I am running inside of a ExecuteGroovyScript
processor (nifi v 1.12.0)
I've been following the examples at
https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-2/ta-p/249018,
and I have something like:
flowFile = session.get()
if(!flowFi
wFile.
Thanks,
Bryan
On Mon, Oct 26, 2020 at 3:49 PM Greene (US), Geoffrey N
wrote:
>
> I have a groovy scripts that I am running inside of a
> ExecuteGroovyScript processor (nifi v 1.12.0)
>
> I’ve been following the examples at
> https://community.cloudera.com/t5/Community
I'm trying to glue two flow files together HORIZONTALLY. That is,
Flowfile1
ID,STARTINGLETTER
1,A
2,B
And flowfile2:
ID, WORD
1,Apple
2, Ball
3, Cat
I want it to become:
ID, STARTINGLETTER, WORD
1,A,Apple
2,B,Ball
3,,Cat
The only way I've been able to figure out how to do this is to write a cus
s less than 2, you can rollback the session and return
(possibly yielding first if you don't want to check again rapidly).
Regards,
Matt
On Tue, Nov 17, 2020 at 4:13 PM Greene (US), Geoffrey N
wrote:
>
> I’m trying to glue two flow files together HORIZONTALLY. That is,
>
> Flowfil
nly want to transfer just the ONE out, since the data was merged in,
but I can manage with two files if I have to make # inputs = # outputs
Thanks
-Original Message-
From: Greene (US), Geoffrey N
Sent: Tuesday, November 17, 2020 8:30 PM
To: users@nifi.apache.org
Subject: RE: [EXTERNAL] Re:
want a call to `session.remove(flowFile1)` instead of transferring it.
Cheers,
Chris Sampson
On Wed, 18 Nov 2020, 20:03 Greene (US), Geoffrey N,
mailto:geoffrey.n.gre...@boeing.com>> wrote:
I've gotten closer with grabbing two files and processing them. I still have
something wro
d. You now
appear to remove 1 and 2 then transfer the 3rd. That should be fine.
Thanks
On Wed, Nov 18, 2020 at 1:50 PM Greene (US), Geoffrey N
mailto:geoffrey.n.gre...@boeing.com>> wrote:
Session.remove()! That’s very helpful, and it makes my numbers come out
correctly. I’m Still
I'm trying to figure out some CI/CM best practices. I want to be able to
design a flow, test the flow on some test data, then distribute that exact same
configuration (definitely flows, probably services, and so on) into
production. I may have multiple engineers working in this environment, a
Im having some trouble with multiple splits/merges. Here's the idea:
Big data -> split 1->Save all the fragment.*attributes into variables -> split
2-> save all the fragment.* attributes
|
Split 1
|
Save fragment.* attributes into split1.fragment.*
|
Split 2
|
Save fragment.* attributes
helps!
-Mark
[1] https://www.youtube.com/watch?v=RjWstt7nRVY
On Feb 24, 2021, at 4:59 PM, Greene (US), Geoffrey N
mailto:geoffrey.n.gre...@boeing.com>> wrote:
Im having some trouble with multiple splits/merges. Here’s the idea:
Big data -> split 1->Save all the fragment.*att
Writing my first ScriptedLookupService in groovy, and I think I have a pretty
simple question:
I'd like to be able to return multiple values in one lookup, but I can't figure
out what my return type needs to be.
String isn't right, obviously, and returning a Map isn't right.
Is lookup only able t
I'm writing a LookupService in Groovy for my ScriptedLookupService. It
is, as everyone had suggested, significantly faster than split/merge.
I'm really very close to having it working. In fact, it works fine when my
is simple; a few strings.
My situation, though, is that my service returns a
Field("allMyIntegers",
RecordFieldType.ARRAY.getArrayDataType(RecordFieldType.INT.getDataType(),
true)))
If I misunderstood what you're trying to do please let me know and I'll help
where I can.
Regards,
Matt
On Fri, Feb 26, 2021 at 7:04 PM Greene (US), Geoffrey N
wrote:
>
>
Brand new installation of nifi-registry here (0.8.0), and nifi 1.12.0, probably
screwed something on the setup.
It's a vanilla installation except that I am
1) Doing this all on windows (I know, I know)
2) Using the corporate gitlab. It uses a self-signed cert, but I put the
cert in
is the problem, but NiFi Registry is not
considered supported on Windows, and there are definitely some issues with the
Git provider on Windows.
https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#system-requirements
Thanks,
Bryan
On Fri, Mar 12, 2021 at 4:12 PM Greene
Im making good progress leaning to use using LookupRecord Processor.
I have it working so that I know how to turn
[{"key": "value1"}, {"key":"value2"}, {"key":"value3"}]
Into
[{"key": "value1","enhanced":1}, {"key":"value2","enhanced":2},
{"key":"value3","enhanced":3}]
BUT
What if you have inner
Sorry for the resend. Thought I'd try one more time. I'm struggling with
LookupRecord.
I'm making good progress leaning to use using Lookup Record Processor.
I have it working so that I know how to turn
[{"key": "value1"}, {"key":"value2"}, {"key":"value3"}]
Into
[{"key": "value1","enhanced":1}
21, at 21:59, Greene (US), Geoffrey N
mailto:geoffrey.n.gre...@boeing.com>> wrote:
Sorry for the resend. Thought I’d try one more time. I’m struggling with
LookupRecord.
I’m making good progress leaning to use using Lookup Record Processor.
I have it working so that I know how to turn
[{“key”
We have a rest endpoint that is "unreliable". It works sometimes.
When it doesn't work, the solution seems to be to sleep for awhile, then try
again
So I put in a retry processor:
http processor<- Retry
| \ ^
Success Failure -|
So far, so good, that loop works. B
Nifi 1.13.2 here
I need to make about 50 REST calls to an interface, but I suspect my rest
service is unreliable.
So I added a ControlRate processor. I tried to slow it down so that I only
process one flowfile/min. I was still getting REST failures, so I decided to
slow it down to only process
I know that in the S3 cli, you can validate an object's integrity during the
upload by passing in the object's MD5 hash.
Does NiFi do that automatically somehow?
Or, post upload, do I need to retrieve the Etag hash from S3 and then compare
it with the hash to a local file somehow?
How do you v
I make heavy use of DistributedMapCacheServer in my nifi flows (one node; not
clustered).
I seem to remember reading that the DistributedMapCacheServer is not to be used
in production; it's a reference implementation only, and it is not really
recommended for production.
Unfortunately, I can n
Some databases (postgres, sql server, others) support native json columns.
With postgres, there's a native jsonb type, with sql server it's a string type,
that you can treat as json.
In any event, once you have the json in the database, one can then query it,
e.g.:
SELECT id,product_name,
in any sort of consistent way. Happy to hear
> your thoughts and input, perhaps we can add some ad-hoc support for
> your use case?
>
> Regards,
> Matt
>
> [1] https://issues.apache.org/jira/browse/NIFI-5901
> [2] https://issues.apache.org/jira/browse/NIFI-5845
> [3] h
So, I came back from two weeks vacation...
My nifi (1.17.0). is misbehaving.
The logs say
2023-07-03 07:04:07,668 ERROR [Index Provenance Events-1]
o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
..
Is this what you are referring to? 20MB string values vs 20 MB document size…
https://github.com/FasterXML/jackson-core/issues/1082
From: Joe Witt
Sent: Monday, September 18, 2023 2:25 PM
To: users@nifi.apache.org
Subject: [EXTERNAL] Re: Large json string field errors
EXT email: be mindful of
What is the minimum or reference implementation of getPropertyDescriptor(name)
in a groovy ScriptedLookup processor?
I don't have any properties, and I had been doing
@Override
PropertyDescriptor getPropertyDescriptor(String name) {
return null
}
But I don't think that is cor
Im having some rather odd behavior on clustering, working on trying to set up
my first cluster (of three nodes).
Im running nifi 1.17.0
everything over ssl, ad I'm using the embedded zookeeper My certs are all
signed by my org, and my truststore DOES have my org's root cert in it. My
cert has
Well, I learned something I thought I'd share:
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#configuration-best-practices
states that you should increase the number of file handles.
The apache guide says to do this, you change limits.conf
Nonetheless, my nifi would run o
I believe you are casting ff.size as an Integer.
8 GB is signifincantly larger than can be held in a 32 bit integer, which is
just 2 GBs.
Suggest you use a double or a big decimal.
Whats happening is you are truncating the high order bits.
From: James McMahon
Sent: Monday, June 17, 2024 7:51
Which is better and why:
1. Using ListSmb/FetchSMB nifi processors to get data from a samba share
2. Mounting the samba share to my linux OS, and using regular ls/fetch File.
Opinions? What are the differences, performance-wise? Are there other
considerations?
Geoff Greene
Associate
Interesting.
The largest number representable by a 32 bit integer is 2147483647.
Which exactly one less than th number you have.
That sounds like a hint...
Sent via the Samsung Galaxy A54 5G, an AT&T 5G smartphone
Original message
From: "Richter, Robert"
Date: 7/11/24 5:38
Ive been introduced to a new situation, thought someone might have some advice.
Imagine a system in which live data is stored in a sql lite database.
The name of the sql lite file changes every hour. The schema is always the
same, but the files roll over every 60 minutes.
Now imagine I have t
33 matches
Mail list logo
