Re: NiFi 2.0.0-M2

2024-03-06 Thread Russell Bateman
Don't know if this is relevant let along helpful, but we were on CentOS for many years and, when it died, we moved to Alma. Best of luck to you, Russ On 3/5/24 07:56, Marton Szasz wrote: CentOS 7 is a very old distribution, so you may run into issues, but in theory, if you can install Java

Re: Block start/stop processors

2024-02-22 Thread Russell Bateman
Isha, Wait, by "lefthand toolbox" are you referring to the process group's toolbar start button or to the start button in the pallet at the upper left of the page? Please clarify. Thanks On 2/22/24 04:39, Isha Lamboo wrote: always use the right-click context menu on a process group you

Re: Issue with Nifi pipeline

2023-11-28 Thread Russell Bateman
Sonia, It sounds like you may prefer the Users Mailing List (us...@nifi.apache.org)rather than this one which is more for custom processors and other development-related activities. Best regards, Russ On 11/27/23 22:49, Sonia Soleimani wrote: Hello, I am working for Telus and there has

Internationalization and localization of the UI and processors

2023-09-29 Thread Russell Bateman
Looking around, I see there have been statements of intent by folk to localize NiFi [1], but few statements as to how far they got. I saw a question on stackoverflow [2] on how to hack relevant Java-annotated references (@CapabilityDescription) which isn't exactly internationalization, but

Custom-processor configuration suggestions

2023-09-27 Thread Russell Bateman
I'm posting this plea for suggestions as I'm short on imagination here. We have some custom processors that need extraordinary amounts of configuration of the sort a flow writer would have to copy and paste in--huge amounts of Yaml, regular expressions, etc. This is what our flow writers are

Re: new PackageFlowFile processor

2023-09-08 Thread Russell Bateman
Uh, sorry, "Version 3" refers to what exactly? On 9/8/23 12:48, David Handermann wrote: I agree that this would be a useful general feature. I also agree with Joe that format support should be limited to*Version 3* due to the limitations of the earlier versions. This is definitely something

Re: Refreshing HTML displayed for View usage

2023-08-21 Thread Russell Bateman
noticed myself, and it's easily reproducible on the Usage pages for processors, is that Nifi is doing a reverse DNS lookup when loading the page. If your DNS is broken, it can take 10-30 seconds for the page content to appear. Kr, On Fri, 18 Aug 2023, 07:29 Russell Bateman, wrote: Matt, I

Re: Refreshing HTML displayed for View usage

2023-08-17 Thread Russell Bateman
and the version is not changing (x.y.z-SNAPSHOT). This is where the browser cache could result in the behavior your seeing. Matt On Thu, Aug 17, 2023 at 3:21 PM Russell Bateman wrote: Thanks, Matt, (Since I use Chrome 99% of the time, I'm using Chrome here.) Clicking on the reload control next

Re: Refreshing HTML displayed for View usage

2023-08-17 Thread Russell Bateman
that tab and retry from the NiFi UI. Let us know if this doesn't help and we can see if something isn't getting generated and updated correctly. Matt On Thu, Aug 17, 2023 at 2:38 PM Russell Bateman wrote: Seems like a really stupid user/browser question, but I cannot seem to get changes I've made

Refreshing HTML displayed for View usage

2023-08-17 Thread Russell Bateman
Seems like a really stupid user/browser question, but I cannot seem to get changes I've made to properties, relationships, attributes read or written, etc. *for custom processors*. Also, from the Properties tab in Configuring Processor, the cartoon blurbs obtained by hovering over (?) aren't

Custom processor once had properties, has no more now, but they still show up

2023-07-20 Thread Russell Bateman
I have a custom processor I modified losing a few properties because a specification change made them useless. I removed them. The processor works, but in configuration for this processor (in the NiFi UI), the processor appears to have kept them, i.e.: they're not disappearing. I would have

Re: Use of attribute uuid and other "native" attributes

2023-07-18 Thread Russell Bateman
latter setting some other attributes as well (see the Developer Guide [1] for more details). Regards, Matt [1]https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html On Tue, Jul 18, 2023 at 12:25 PM Russell Bateman wrote: I have a custom processor, /SplitHl7v4Resources/, that splits

Use of attribute uuid and other "native" attributes

2023-07-18 Thread Russell Bateman
I have a custom processor, /SplitHl7v4Resources/, that splits out individual FHIR resources (Patients, Observations, Encounters, etc.) from great Bundle flowfiles. So, for a given flowfile, it's split into hundreds of smaller ones. When I do this, I leave the existing NiFi attributes as they

Re: Possible Docker bug in 1.13.2

2023-06-08 Thread Russell Bateman
SDJT/p1686135561932289?thread_ts=1686135561.932289=CDGMCSDJT On Thu, 8 Jun 2023, 00:10 Russell Bateman, wrote: I'm re-rolling in order to update the Java inside to 11 in order to permit using the new Java HTTP client. This seems to work well; I fixed the bug locally. Maybe too old to be impor

Re: Possible Docker bug in 1.13.2

2023-06-07 Thread Russell Bateman
, until we can shed the old NAR we cannot rebuild, we're stuck at NiFi 1.13.2. On 6/7/23 15:31, Russell Bateman wrote: I downloaded sources to 1.13.2 in order to hand-spin my own container image. When I got down to /nifi-1.13.2/nifi-docker/dockerhub/Dockerfile/, I found: ... ARG

Possible Docker bug in 1.13.2

2023-06-07 Thread Russell Bateman
I downloaded sources to 1.13.2 in order to hand-spin my own container image. When I got down to /nifi-1.13.2/nifi-docker/dockerhub/Dockerfile/, I found: ... ARG NIFI_VERSION=1.13.1 ... and the version is also wrong in /DockerImage.txt/ which /DockerBuild.sh/ consumes. Indeed, the

Re: Usage Documentation for Custom Processors

2023-04-04 Thread Russell Bateman
Matthew, If you feel that the documentation generated from the annotations at the top of your custom processor class (@CapabilityDescription, etc., of which Bryan spoke) is insufficient, it's also possible to supplement it with

Re: ReplaceText 1.19 not processing Flowfile

2022-12-11 Thread Russell Bateman
Also, this strikes me as a NiFi Users List (us...@nifi.apache.org) question though many of us haunt both forums. On 12/11/22 07:55, Mark Payne wrote: Hello, It looks like the attachment didn’t come through. The mailing list often strips out attachments from emails. Perhaps put them in

Re: Grammar error in error message

2022-11-16 Thread Russell Bateman
Yes it should be the infinitive */receive/* instead of the past participle /received/. On 11/16/22 08:18, Paul Schou wrote: This error message does not look like it is grammatically correct:

Re: How to manage security artifacts from a custom processor

2022-07-05 Thread Russell Bateman
representing the third-party service in a truststore and maybe another (a private key) in a keystore. On 7/5/22 16:30, Russell Bateman wrote: From a custom processor, I intend to interface with a third-party service (via simple HTTP client), however, I would need as I understand it to a) maintain

How to manage security artifacts from a custom processor

2022-07-05 Thread Russell Bateman
From a custom processor, I intend to interface with a third-party service (via simple HTTP client), however, I would need as I understand it to a) maintain a private key by which I can identify myself to that third-party service and b) maintain a trusted-store certificate by which I

Re: Reg Nifi java code generation

2022-05-02 Thread Russell Bateman
You don't have to write Java code to benefit from NiFi which is an insanely useful framework all by itself with jillions of super-useful processors ready for use. However, if you plan to code your own, proprietary processor to do something that hasn't been covered, here's a likely place to

Re: [DISCUSS] NiFi 2.0 Release Goals

2021-07-23 Thread Russell Bateman
Russell Bateman wrote: Joe, I apologize for the off-topic intrusion, but what replaces templates? The Registry? Templates rocked and we have used them since 0.5.x. Russ On 7/23/21 8:31 AM, Joe Witt wrote: David, I think this is a highly reasonable approach and such a focus will greatly help make

Re: [DISCUSS] NiFi 2.0 Release Goals

2021-07-23 Thread Russell Bateman
Joe, I apologize for the off-topic intrusion, but what replaces templates? The Registry? Templates rocked and we have used them since 0.5.x. Russ On 7/23/21 8:31 AM, Joe Witt wrote: David, I think this is a highly reasonable approach and such a focus will greatly help make a 2.0 release

Re: Penalizing one part of a flow over another

2021-04-22 Thread Russell Bateman
of maybe 10 MB/sec into the sub-flow. Again, that could cause backpressure so you’d want to consider FlowFile Expiration if you’d rather lose the FlowFiles than allow them to affect the main flow. Hope that’s helpful! Thanks -Mark On Apr 22, 2021, at 9:44 AM, Russell Bateman wrote: I have

Penalizing one part of a flow over another

2021-04-22 Thread Russell Bateman
I have a flow performing ETL of HL7v4 (FHIR) document on their way to indexing and storage. Custom processors perform the important transformations. Performance of this flow is at a premium for us. At some point along the way I want to gate off copies of raw or of transformed FHIR records (the

Re: [DISCUSS] Processors Market

2021-03-24 Thread Russell Bateman
Javi, Don't despair. Could just be that folk are busy and haven't had time to reflect upon it. On 3/23/21 11:55 PM, Javi Roman wrote: I see that it has not been well received, I thought it would be a good idea :-( -- Javi Roman Twitter: @javiromanrh GitHub: github.com/javiroman Linkedin:

Re: Preconfiguring dynamic properties

2021-02-24 Thread Russell Bateman
. On Wed, Feb 24, 2021 at 12:02 PM Russell Bateman wrote: I have a dynamic property in a custom processor that my down-streamers struggle a little bit to configure (requires newlines and a peculiar format). I would like to "preconfigure" a dynamic property as an example that they

Preconfiguring dynamic properties

2021-02-24 Thread Russell Bateman
I have a dynamic property in a custom processor that my down-streamers struggle a little bit to configure (requires newlines and a peculiar format). I would like to "preconfigure" a dynamic property as an example that they can either modify or erase to add their own. Most of them would

Re: [discuss] we need to enable secure by default...

2021-02-10 Thread Russell Bateman
I second the concerns expressed, but second especially Bryan's pointing out that requiring LDAP/AD to be set up in order even to begin to use our framework would be a bit onerous for developers just interested in getting work done and a barrier to considering the framework should it be erected

Re: java api for changing parameter context

2021-01-27 Thread Russell Bateman
Wait! Can't this be done using the ReST APIs? On 1/27/21 3:24 AM, u...@moosheimer.com wrote: Hello NiFi-Core-Team, Are you planning to create a high-level Java API for setting (and clearing) individual parameters in the parameter context, so we can use this API in processor development?

Re: Static processor design

2021-01-09 Thread Russell Bateman
:09 AM, Mark Bean wrote: Russell, You can use "session.get(N)" where N is an integer. This will get up to N flowfiles per OnTrigger() call. -Mark On Fri, Jan 8, 2021 at 5:07 PM Russell Bateman wrote: Very well, I have decided to force customer flowfiles through this processor (I

Re: Static processor design

2021-01-08 Thread Russell Bateman
Very well, I have decided to force customer flowfiles through this processor (I did check out the /Listen/* processors, but chose this easier solution). This now works. However, It brings up another question: is this the most efficient way to pass flowfiles straight through this processor

Re: Static processor design

2021-01-08 Thread Russell Bateman
I only put the code I want to execute in onTrigger(), I suspected it would not fire there. I know that this isn't what processors do. Configuration is a messy problem to solve when your downstreamers want it made easy. This is supposed to be a solution that allows them to remain in the NiFi UI

Re: Static processor design

2021-01-08 Thread Russell Bateman
statement from onPropertyModified()does reach the log every time I change properties to remove old or introduce new properties). On 1/7/21 6:38 PM, Russell Bateman wrote: (Inadequate title; didn't know what to call it.) I have written a processor that doesn't feature any relationships.

Static processor design

2021-01-07 Thread Russell Bateman
(Inadequate title; didn't know what to call it.) I have written a processor that doesn't feature any relationships. It accepts dynamically properties that, in theory, when created (or removed, or values added or changed), and sets data into a class inside my NAR. I wonder, however, at what

Re: Safely updating custom processors in existing flows...

2020-12-30 Thread Russell Bateman
I will be very happy to read what others contribute. Chris' suggestion, which has huge implications, I think, for my IDE and project structure, will be something to look into. However, I was hoping someone would articulate the difference between *property names* and *property display names*,

Safely updating custom processors in existing flows...

2020-12-30 Thread Russell Bateman
I have a custom processor actively used in customer flows. I need to enhance it, but avoid invalidating it when I update custom's existing NiFi installations. I know that the processor properties, in particular, the conflict between the property .name()and .displayName(), is a good way

Re: ETL to Nifi Migration

2020-12-26 Thread Russell Bateman
Unless you're certain you will need to write custom processors for NiFi, the forum you really want to subscribe to and post in is NiFi Users Best regards! On 12/26/20 7:37 AM, Sumith Karthikeyan wrote: Hi Team, Hope you all doing well !!! This regards a Middle East public sector

Re: Okay to manage ComponentLog via ThreadLocal?

2020-12-08 Thread Russell Bateman
(); logger.warn( “This is a warning!” ); } } Perhaps I’m missing something - or perhaps you made things simpler than they really are for demonstration purposes? Thanks -Mark On Dec 8, 2020, at 4:54 PM, Russell Bateman wrote: Because it's so onerous to pass a reference to the logger down through

Okay to manage ComponentLog via ThreadLocal?

2020-12-08 Thread Russell Bateman
Because it's so onerous to pass a reference to the logger down through parameters lists, I thought I might try using Java's thread-local store. I haven't been using it for anything else either, but I thought I'd start. For now, the logger is the only thing that tempts me. In past lives as a

Re: APACHE NIFI CLUSTER INSTALLATION WALKTHROUGHS FOR 2NODE ON LINUX OS

2020-09-23 Thread Russell Bateman
You have treated Google as your friend and worked through the dozen or so examples by other folk doing this, right? --just a suggestion. On 9/23/20 1:21 PM, Abiodun Adegbile wrote: */Hello Team,/* Still looking forward to your reply. / / /I got this though after i tried setting up the

Re: APACHE NIFI CLUSTER INSTALLATION WALKTHROUGHS FOR 2NODE ON LINUX OS

2020-09-20 Thread Russell Bateman
Google is your friend. https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.5.1/nifi-configuration-best-practices/content/basic-cluster-setup.html https://bryanbende.com/development/2018/10/23/apache-nifi-secure-cluster-setup https://mintopsblog.com/2017/11/12/apache-nifi-cluster-configuration/

Re: TestRunner: enqueueing multiple flowfiles

2020-08-31 Thread Russell Bateman
i-mock/src/main/java/org/apache/nifi/util/StandardProcessorTestRunner.java#L443-L448 Instead of creating a new MockProcessSession, you would get the ProcessSessionFactory from the TestRunner and then call createProcessSession(). On Mon, Aug 31, 2020 at 9:01 AM Russell Bateman wrote: In my JUn

TestRunner: enqueueing multiple flowfiles

2020-08-31 Thread Russell Bateman
In my JUnit testing of a custom processor, I need to queue up at least two flowfiles. I see that there is an implementation of TestRunner.enqueue()that takes*a list of flowfiles*, but I'm used to using the implementation of this method that creates me a flowfile from bytes or a stream. I do

Re: From one flowfile to two...

2020-08-27 Thread Russell Bateman
or explain the experience as well. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com He/Him PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On Aug 27, 2020, at 10:10 AM, Russell Bateman wrote: I needed to get back here... I took this advice to heart and finished

Re: From one flowfile to two...

2020-08-27 Thread Russell Bateman
Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On Aug 27, 2020, at 10:10 AM, Russell Bateman wrote: I needed to get back here... I took this advice to heart and finished my processor. Thanks to Matt and Mark for all their suggestions! They cleared up a few things. There was one bug

Re: From one flowfile to two...

2020-08-27 Thread Russell Bateman
, the NiFi mailing lists are a place both for great information and being treated well. Russ On 8/25/20 12:24 PM, Mark Payne wrote: Russ, Several comments here. I’ve included them inline, below. Hope it’s helpful. Thanks -Mark On Aug 25, 2020, at 2:09 PM, Russell Bateman wrote: Thanks for your

Re: From one flowfile to two...

2020-08-25 Thread Russell Bateman
ich is the same of course) and overwriting it on the way out. Regards, Matt On Mon, Aug 24, 2020 at 6:37 PM Russell Bateman wrote: I am writing a custom processor that, upon processing a flowfile, results in two new flowfiles (neither keeping the exact, original content) out two different relatio

From one flowfile to two...

2020-08-24 Thread Russell Bateman
I am writing a custom processor that, upon processing a flowfile, results  in two new flowfiles (neither keeping the exact, original content) out two different relationships. I might like to route the original flowfile to a separate relationship. FlowFile original = session.get(); Do I need

Re: Suggestions for splitting, then reassembling documents

2020-08-21 Thread Russell Bateman
, Jason -- Jason C. Sherman, CSSLP, CISSP Owner Logical Software Solutions, LLC Solid. Secure. Software. http://logicalsoftware.co/ .co? Yes, your data isn't always what you expect. We'll make sense of it. https://www.linkedin.com/in/lss-js/ On Tue, Aug 18, 2020 at 12:38 PM Russell Bateman wrote

Suggestions for splitting, then reassembling documents

2020-08-18 Thread Russell Bateman
I am writing custom processors that juggle medical documents (in a more or less proprietary format). The document are always XML and contain two, major parts: 1. an original document which may be text, HL7v2 or XML and may contain HTML between ... , could be many megabytes in size 2. XML

Live development of custom processors and JAVA_HOME--confirm best practice

2020-08-13 Thread Russell Bateman
When installing NiFi in production, Ansible can be used to set up JAVA_HOME. There is zero problem for users of NiFi. However, from a development host, given the now rapid cadence of Java releases, we sometimes run into problems launching a private installation of NiFi in the course of

Re: Need to know if there is multiple template support

2020-08-06 Thread Russell Bateman
Since the advent of the NiFi Registry, templates are sort of deprecated. The Registry is a brilliant design far better able to support what you seem to be asking for. By the way, this is much more a "user" question you might have asked in that forum rather than this "dev" forum. Cheers! On

Re: Failing to update custom processor properties names, displayNames, etc.

2020-07-18 Thread Russell Bateman
code is not being properly loaded/used by NiFi’s server application. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com He/Him PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On Jul 17, 2020, at 3:48 PM, Russell Bateman wrote: If I have changed a custom

Failing to update custom processor properties names, displayNames, etc.

2020-07-17 Thread Russell Bateman
If I have changed a custom processor's PropertyDescriptor.nameand/or .displayName,including changes I have made to my /additionalDetails.html, /and I have: - removed that processor from my test flow or removed flow.xml.gz altogether - removed my NAR from /${NIFI_ROOT}///custom-lib/ and bounced

Re: Derby as DBCP service, error from Kerberos?

2020-07-14 Thread Russell Bateman
ot;. On Tue, Jul 14, 2020 at 10:24 AM Russell Bateman wrote: Thanks for the responses. I did have this dependency already before mailing to the forum: 1.11.0 ... org.apache.nifi nifi-kerberos-credentials-service ${nifi.version} Oth

Re: Derby as DBCP service, error from Kerberos?

2020-07-14 Thread Russell Bateman
ency in your project... https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-dbcp-service/pom.xml#L50-L55 On Mon, Jul 13, 2020 at 5:24 PM Russell Bateman wrote: I'm trying to use Apache Derby as the DBCP controller in JUnit tests. For the fir

Derby as DBCP service, error from Kerberos?

2020-07-13 Thread Russell Bateman
I'm trying to use Apache Derby as the DBCP controller in JUnit tests. For the first test, I start off vetting my ability to inject Derby as the DBCP controller I want to use. But, right off, I get this Kerberos error. I wasn't trying to use Kerberos, but maybe I'm missing configuration to tell

Re: Difficulty using DBCPService

2020-06-25 Thread Russell Bateman
. That interface is in nifi-standard-services-api-nar, which is why you need it as a parent NAR to your custom NAR. Regards, Matt On Mon, Jun 22, 2020 at 7:52 PM Russell Bateman wrote: Putting this dependency in the root, with a version, worked. My processor is loading. Have you the time to tell

Re: Difficulty using DBCPService

2020-06-22 Thread Russell Bateman
tried and looked. I was already checking on that. Maybe I was fooling myself. Thanks, Matt On 6/22/20 5:43 PM, Matt Burgess wrote: You’ll want to add a 1.12.0-SNAPSHOT (or a released version of NiFi) before the ending tag Sent from my iPhone On Jun 22, 2020, at 7:33 PM, Russell Bateman wrote

Re: Difficulty using DBCPService

2020-06-22 Thread Russell Bateman
as a parent? That should be where DBCPService is defined Sent from my iPhone On Jun 22, 2020, at 6:30 PM, Russell Bateman wrote: I find myself obliged to pick back up a custom processor, written by someone else a few years ago (in the NiFi 0.7.x era) at my company, that makes use of DBCPService

Difficulty using DBCPService

2020-06-22 Thread Russell Bateman
I find myself obliged to pick back up a custom processor, written by someone else a few years ago (in the NiFi 0.7.x era)  at my company, that makes use of DBCPService. While I think I understand the nuances of interface versus concrete controller class, etc. I probably need a push out the

Re: Nifi tutorials

2020-05-26 Thread Russell Bateman
Certainly! On 5/26/20 12:32 PM, Anuj Jain wrote: Hi Team, I am currently working on Apache-Nifi in my company from past 1 year. I am planning to launch a series of videos for Nifi learners about what I have learned in Nifi in last 1 year on youtube. Am I allowed to do that? Regards, Anuj

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
); } This seems very close to working for me; I don't see anything wrong and just need to plug in my SAX parser. This modified session is a new pattern for me (and a useful one). Thanks! On 3/31/20 12:44 PM, Russell Bateman wrote: (Oh, I see where *out*comes from, but not *modified*.) On 3/31/20

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
(Oh, I see where *out*comes from, but not *modified*.) On 3/31/20 12:35 PM, Russell Bateman wrote: Wait, where is *modified*from? Thanks On 3/31/20 12:24 PM, Mark Payne wrote: Russ, OK, so then I think the pattern you’d want to follow would be something like this: FlowFile original

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
And, also *out*? On 3/31/20 12:35 PM, Russell Bateman wrote: Wait, where is *modified*from? Thanks On 3/31/20 12:24 PM, Mark Payne wrote: Russ, OK, so then I think the pattern you’d want to follow would be something like this: FlowFile original = session.get(); if (flowFile == null

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
in) { processWithSaxParser(in, out); } }); } }); session.transfer(output, REL_SUCCESS); session.remove(original); Thanks -Mark On Mar 31, 2020, at 2:04 PM, Russell Bateman wrote: Mark, Thanks for getting back. My steps are: 1. Read the "first half" of the in

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
ionship or whatever makes sense for you. Hope this helps! -Mark On Mar 30, 2020, at 4:23 PM, Russell Bateman <mailto:r...@windofkeltia.com>> wrote: If I haven't worn out my welcome, here is the simplified code that should demonstrate either that I have miscoded your suggestions

Re: Reading the incoming flowfile "twice"

2020-03-31 Thread Russell Bateman
when doing the non-sax part? If you consume the stream right up to the sax part ( the stream POS is at the start of the xml ) then you can just pass the stream to sax as is can’t you? On March 30, 2020 at 16:23:27, Russell Bateman (r...@windofkeltia.com) wrote: If I haven't worn out my welcome

Re: Reading the incoming flowfile "twice"

2020-03-30 Thread Russell Bateman
. On Sun, Mar 29, 2020 at 12:16 AM Russell Bateman wrote: Joe and Mike, Sadly, I was not able to get very far on this. It seems that the extend to which I copy the first half of the contents of the input stream, I lose what comes after when I try to read again, basically, the second half comprising

Re: Reading the incoming flowfile "twice"

2020-03-29 Thread Russell Bateman
ft off the code that prints, "And now, we'll go on to the SAX > > parser..." It's in the next flowfile = session.write( ... ). I have unit > > tests that verify the good functioning of > > copyCxmlHeaderAndDocumentToOutput(). The SAX error occurs because the > > "file

Re: Reading the incoming flowfile "twice"

2020-03-28 Thread Russell Bateman
in if you can, Russ On 3/27/20 3:08 PM, Joe Witt wrote: you should be able to call write as many times as you need. just keep using the resulting flowfile reference into the next call. On Fri, Mar 27, 2020 at 5:06 PM Russell Bateman wrote: Mike, Many thanks for responding. Do you mean to s

Re: Reading the incoming flowfile "twice"

2020-03-27 Thread Russell Bateman
. just keep using the resulting flowfile reference into the next call. On Fri, Mar 27, 2020 at 5:06 PM Russell Bateman wrote: Mike, Many thanks for responding. Do you mean to say that all I have to do is something like this? public void onTrigger( final ProcessContext context, final

Re: Reading the incoming flowfile "twice"

2020-03-27 Thread Russell Bateman
ut them into temporary memory, say, a ByteArrayOutputStream. Russ On 3/27/20 10:03 AM, Mike Thomsen wrote: session.read(FlowFile) just gives you an InputStream. You should be able to rerun that as many times as you want provided you properly close it. On Fri, Mar 27, 2020 at 11:25 AM Russell B

Reading the incoming flowfile "twice"

2020-03-27 Thread Russell Bateman
In my custom processor, I'm using a SAX parser to process an incoming flowfile that's in XML. Except that, this particular XML is in essence two different files and I would like to split, read and process the first "half", which starts a couple of lines (XML elements) into the file) not using

Re: Managing custom processor logging statements during JUnit testing

2020-03-10 Thread Russell Bateman
On Mar 6, 2020, at 12:13 PM, Russell Bateman wrote: I'm interested in getting log statements that may occur from my *custom processor* code at the INFO, DEBUGand TRACElevels when running unit tests for it. This means that I would like to set (programmatically at runtime or by configuration) wha

Managing custom processor logging statements during JUnit testing

2020-03-06 Thread Russell Bateman
I'm interested in getting log statements that may occur from my *custom processor* code at the INFO, DEBUGand TRACElevels when running unit tests for it. This means that I would like to set (programmatically at runtime or by configuration) what I am used to setting in

Re: How to preclude user-defined properties...

2020-02-25 Thread Russell Bateman
something that could be improved, I believe. Thanks -Mark On Feb 25, 2020, at 3:18 PM, Russell Bateman wrote: ...in a custom processor. I have a custom processor (that I wrote) and, in on-canvas configuration, the dialog allows the user to create custom properties. I have no need of any

How to preclude user-defined properties...

2020-02-25 Thread Russell Bateman
...in a custom processor. I have a custom processor (that I wrote) and, in on-canvas configuration, the dialog allows the user to create custom properties. I have no need of any and wish to help my down-streamers by removing the option of creating any user-defined properties. How is this

Re: additionalDetails.html styles

2020-02-22 Thread Russell Bateman
/nifi-hbase-bundle/nifi-hbase-processors/src/main/resources/docs/org.apache.nifi.hbase.PutHBaseCell/additionalDetails.html On Thu, Feb 20, 2020 at 12:20 PM Russell Bateman wrote: I would like to get what I'm offering in /additionalDetails.html/ for my custom processor to use the same font style

additionalDetails.html styles

2020-02-20 Thread Russell Bateman
I would like to get what I'm offering in /additionalDetails.html/ for my custom processor to use the same font style used down the left-hand column listing processors (under /NiFi Documentation/). I'm not an idiot when it comes to HTML/CSS, however, I have been unable in experimenting to see

Re: Need to have my email added

2020-02-20 Thread Russell Bateman
Eric, If I understand what you're asking, you just need to sign up properly. Check out community information at https://nifi.apache.org/mailing_lists.html . Best regards. On 2/20/20 9:23 AM, Butts, Eric wrote: I need to have my email added to forum discussion buttse...@hotmail.com

Re: View Usage brings up (default) Apache NiFi Overview documentation

2020-01-29 Thread Russell Bateman
, 2020, at 1:49 PM, Russell Bateman wrote: Andy, I have formulated, on the basis of posts I've seen over the years, that there are two ways to build a NAR containing one's own, custom processors, what I term a /single-module product/ and a /multi-module product/. mostly based on whether I nee

Re: View Usage brings up (default) Apache NiFi Overview documentation

2020-01-28 Thread Russell Bateman
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 On Jan 27, 2020, at 2:30 PM, Russell Bateman wrote: Addendum: the @TagsI add are not showing up in the tag field at the left, but when I scroll down in the processor list and see my new custom processor, I do see these tags

Re: View Usage brings up (default) Apache NiFi Overview documentation

2020-01-27 Thread Russell Bateman
Addendum: the @TagsI add are not showing up in the tag field at the left, but when I scroll down in the processor list and see my new custom processor, I do see these tags listed next, to its right. On 1/27/20 3:27 PM, Russell Bateman wrote: Perhaps I made my question less clear that I could

Re: View Usage brings up (default) Apache NiFi Overview documentation

2020-01-27 Thread Russell Bateman
for my capability description, properties, relationships, etc. Also, I am using the @Tags annotation to add a couple of quick-find tags: neither of these are showing up when I go to create an instance of the processor. This is also likely a clue to what I'm doing wrong. Thanks, Russ On 1/24/20 3:30

View Usage brings up (default) Apache NiFi Overview documentation

2020-01-24 Thread Russell Bateman
My custom processor's usage, which should come from the @CapabilityDescription annotation of the class containing the onTrigger() method, nevertheless is nothing more than the Apache NiFi Overview. I am able to place an instance on the canvas via the processor Component Tool (so, the custom

Re: Unable to update processor...

2020-01-14 Thread Russell Bateman
-lib/ subdirectory or something. On 1/13/20 2:44 PM, Russell Bateman wrote: I have a custom processor that I have updated (in potentially breaking ways and I know it), but I cannot seem to make NiFi 1.10.0 open its mouth and swallow it. I have copied my NAR file to /1.10.0/custom-lib

Unable to update processor...

2020-01-13 Thread Russell Bateman
I have a custom processor that I have updated (in potentially breaking ways and I know it), but I cannot seem to make NiFi 1.10.0 open its mouth and swallow it. I have copied my NAR file to /1.10.0/custom-lib/; there is no other version (in the /lib/ subdirectory for example). When I bounce

NiFi 1.10.0 JRE clarification...

2019-11-06 Thread Russell Bateman
Looking at the release notes, I see: * Apache NiFi can now be built on either Java 8 or Java 11! When built on Java 8 it can run on Java 8 or Java 11. The implications of this statement make me ask: * At https://nifi.apache.org/download.html, how is it built? JDK 11? * Can I run it using

Re: [DISCUSS] Time based release cycles

2019-11-05 Thread Russell Bateman
Kafka is first-rate, rock-star technology, just as is NiFi. It would be nice to find something from Kafka elaborating on how this regular and accelerated release cadence is working out for them, how much more work it's been, what problems they've experienced, etc. I show their releases over

Re: Azure Event Hub Processors Upgrade Problem

2019-08-22 Thread Russell Bateman
I can probably help, Sunny. I made some notes a few years ago on doing this. How this is done on later versions hasn't changed I don't think. Check out http://www.javahotchocolate.com/notes/nifi.html#20160323. Russ Bateman On 8/22/19 11:12 AM, Sunny Zhang wrote: Hi there, I’m a dev from

Re: Java 11 build support is live!

2019-08-15 Thread Russell Bateman
Yes, at the risk of adding nothing more than a "me too," I nevertheless wish to add my thanks. Good job! On 8/15/19 12:04 PM, Jeff wrote: Apache NiFi Developers, Apache NiFi can now be built with Java 11! Thanks to those that helped with the review and committing of several PRs to get us to

StateManager race condition potential

2019-08-09 Thread Russell Bateman
I'm assuming that the StateManagerprotects itself against race conditions for the consuming (custom) processor, but I'd like confirmation on that. Let's say something simple like we get an integer out of state to which we can add one to get the next (piece of work to do), then immediately bump

Re: [board report] Apache NiFi - July 2019

2019-07-10 Thread Russell Bateman
I mulled over whether it's appropriate to ask within this context. I guess I'll ask. Slap me if I have badly chosen, but where is NiFi respective to "modern" JDK versions? Is that too much detail for the audience of this periodic report? Thanks. On 7/10/19 11:15 AM, Joe Witt wrote: Team, I

Re: nifi

2019-07-05 Thread Russell Bateman
It's short for "Niagra files" in the sense that you're meant to think of huge flows of files like water over the falls . On 7/5/19 6:00 AM, Arpad Boda wrote: "nye fye" (nī fī) is the preferred pronunciation. https://nifi.apache.org/faq.html On

NAR extensions warning during build

2019-05-30 Thread Russell Bateman
In /pom.xml/, I have:   *org.apache.nifi*   *nifi-nar-maven-plugin*   *3.5.1*   *true* I get this, which isn't fatal, when my NAR is built. I would like a) to know to what "extension(s)" this refers (related to the plug-in configuration in /pom.xml/?), b) where to put extension

Latest NiFi customs?

2019-04-17 Thread Russell Bateman
After a couple of years absence from NiFi (prior to Java 9), I find myself just now back in a developer role in a company that uses NiFi. (This is a pleasant thought, I might add, as I believe that NiFi rocks.) I have inherited an existing implementation that's sorely aged and, though I've

Re: Calling getLogger() from @OnScheduled, @OnStopped, etc.

2018-04-12 Thread Russell Bateman
to the variable. Generally you should just call getLogger() whenever it is needed, or you could assign it to a variable from inside OnScheduled. On Thu, Apr 12, 2018 at 5:28 PM Russell Bateman <r...@windofkeltia.com> wrote: Thanks for responding, Andy. I am able to use it, li

  1   2   3   >