Re: Secure Nifi with DigiCert

[Andy LoPresto]

If this is the same question posted to the Slack channel earlier, I’ll reply 
here as well. 

Importing the .p12 file into your browser provides the client certificate 
identifying you as a user to the site. When you visit google.com, only one end 
of the connection (Google, the server) presents a certificate, which you the 
user (your browser) verify and decide to trust. When you visit a NiFi instance 
which is secured and has no other authentication mechanism configured, the only 
way to authenticate is to present a client certificate.


Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On May 19, 2020, at 7:24 PM, Ren Yang  wrote:
> 
> 
> 
>> Hi Nifi Team,
>> Thanks for reading my email. I have encountered an issue of securing Nifi 
>> with Digicert issue. Could you please read the following details.
>>  
>> I have got the Digicert related files and generated the keystore.jks and 
>> truststore.jks. And all other setup steps have finished. However, when I 
>> come to my nifi site with HTTPS URL, it denied.
>> 
>>  
>> Next, I double clicked the nifi.p12 which generated by openssl command, 
>> imported it into Keychain access. 
>>  
>> 
>>  
>> Then I access my Nifi https url again, the cert confirmation window comes. 
>> After pressed “OK”, I arrived the Nifi home page. My question is why I need 
>> to manfully import the .p12 file into browser. Hasn’t it been working like 
>> any other public websites (such as https://www.google.com 
>> ) without doing anything on client side?
>>  
>> 
>>  
>>  
>> Please let me know if you have any questions. Awaiting for your reply. Thank 
>> you!
>>  
>> 
> 
> 
> 
> 
> Ren Yang
> ryang...@gmail.com 
> 
> 
> 

Re: Combine Attributes & Content

[Matt Burgess]

Dweep,

Depending on how complex the content JSON is, you might be able to use
ReplaceText to smuggle the attributes into the text, but this can be
tricky as you need to match on the opening JSON and the rest, and then
replace it with the opening JSON, the attributes, then the rest in
order to preserve valid JSON.

You may also be able to use UpdateRecord to add fields to your
content, I believe if the Replacement Value Strategy property is set
to "Literal Value" you can use Expression Language which may be able
access flow file attributes.

An alternative is to use a scripted processor if you're comfortable
with a scripting language such as Groovy, as it can "slurp" JSON into
a Plain Old Groovy Object (POGO), then you can add fields and
serialize back out to JSON. I have a somewhat similar example here
[1].

Regards,
Matt

[1] 
http://funnifi.blogspot.com/2016/02/executescript-json-to-json-conversion.html

On Tue, May 19, 2020 at 10:46 AM Dweep Sharma  wrote:
>
> Hi All,
>
> Does anyone have a strategy to combine Attributes & Content into a single 
> Flowfile?
>
> AttributesToJson seems like the best way convert all attributes to content 
> -Json but I
>
> lose the original content.
>
> Basically, I have a HandleHTTPRequest Processor and headers and body must be 
> combined,
>
> currently Headers are stored as Attributes in a flowfile
>
> -Dweep
>
>
> ::DISCLAIMER::
> 
>
> The contents of this e-mail and any attachments are confidential and intended 
> for the named recipient(s) only.E-mail transmission is not guaranteed to be 
> secure or error-free as information could be intercepted, corrupted,lost, 
> destroyed, arrive late or incomplete, or may contain viruses in transmission. 
> The e mail and its contents(with or without referred errors) shall therefore 
> not attach any liability on the originator or redBus.com. Views or opinions, 
> if any, presented in this email are solely those of the author and may not 
> necessarily reflect the views or opinions of redBus.com. Any form of 
> reproduction, dissemination, copying, disclosure, modification,distribution 
> and / or publication of this message without the prior written consent of 
> authorized representative of redbus.com is strictly prohibited. If you have 
> received this email in error please delete it and notify the sender 
> immediately.Before opening any email and/or attachments, please check them 
> for viruses and other defects.

Combine Attributes & Content

[Dweep Sharma]

Hi All,

Does anyone have a strategy to combine Attributes & Content into a single
Flowfile?

AttributesToJson seems like the best way convert all attributes to content
-Json but I

lose the original content.

Basically, I have a HandleHTTPRequest Processor and headers and body must
be combined,

currently Headers are stored as Attributes in a flowfile

-Dweep

-- 
*::DISCLAIMER::




The contents of this e-mail and any attachments are confidential and 
intended for the named recipient(s) only.E-mail transmission is not 
guaranteed to be secure or error-free as information could be intercepted, 
corrupted,lost, destroyed, arrive late or incomplete, or may contain 
viruses in transmission. The e mail and its contents(with or without 
referred errors) shall therefore not attach any liability on the originator 
or redBus.com. Views or opinions, if any, presented in this email are 
solely those of the author and may not necessarily reflect the views or 
opinions of redBus.com. Any form of reproduction, dissemination, copying, 
disclosure, modification,distribution and / or publication of this message 
without the prior written consent of authorized representative of redbus. 
com is strictly prohibited. If you have received this 
email in error please delete it and notify the sender immediately.Before 
opening any email and/or attachments, please check them for viruses and 
other defects.*

Re: Creating parameters from exported template?

[Bryan Bende]

Hi Nathan,

Right now the git repo behind registry is mostly write-only, meaning if you
add something to it outside of going through the given NiFi Registry
instance, it won't be visible.

The only exception to this is when starting a brand new registry that has
an empty metadata database. So if you ensure the DB is new/wiped every time
that you export the git repo to production, then when you start registry
the first time it will rebuild the database from the git repo and it should
work.

Thanks,

Bryan

On Tue, May 19, 2020 at 8:46 AM  wrote:

> Hi Bryan,
>
>
>
> Would the same also work by using two separate git persistence providers
> pointed at two different git repositories? We have a remote production
> instance with no remote access (effectively airgapped).
>
>
>
> Our plan is when we deploy to production, provide a full exported copy of
> the git repository, and it’s history to a locally running instance of a git
> server in production. Which we will then configure the production NiFi
> Registry to use that export of the git repository for read-only operations.
>
>
>
> So two separate Nifi Registries with two independent git repo’s, the
> production one as a cloned export of the Dev/Test Git Repo.
>
>
>
> We are planning on testing this within the next few weeks.
>
>
>
> Kind Regards,
>
>
>
> Nathan
>
>
>
> *From:* Bryan Bende [mailto:bbe...@gmail.com]
> *Sent:* 19 May 2020 15:36
> *To:* users@nifi.apache.org
> *Subject:* Re: Creating parameters from exported template?
>
>
>
> John,
>
>
>
> You can have a nifi registry on each network and use Nifi CLI to export
> and import the flows between registries. There is one limitation in that
> nested versioning doesn’t currently work when doing that.
>
>
>
> Also, the parameter contexts do live outside the flow at the controller
> level, however in order to use one of them, the use needs read access to
> the context and write access to the process group. So you can setup fine
> grained policies on each context to restrict who can use it.
>
>
>
> Thanks,
>
>
>
> Bryan
>
>
>
> On Tue, May 19, 2020 at 8:26 AM Bryan Bende  wrote:
>
> The best option for moving flows across environments would be versioned
> flows in Nifi registry, which have full support for parameters.
>
>
>
> On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin 
> wrote:
>
> Hum, did not see that, and because I am still using "old" (1.9.2) version
> where parameters are not available.
>
>
>
> But I understand like you
>
>
>
> Le mar. 19 mai 2020 à 13:56, James  a écrit :
>
> Hi
>
> Thanks for the response.
>
> According to this, it's not recommended:
> https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables
>
> "Variables and the nifi.variable.registry.properties property will be
> removed in a future release. As a result, it is highly recommended to
> switch to Parameters."
>
> Or am I thinking of the wrong thing?
>
> thanks
>
> On 2020/05/19 08:11:14, Etienne Jouvin  wrote:
> > Hi.
> >
> > In this case why don't you use variables ?
> >
> > Etienne
> >
> >
> > Le mar. 19 mai 2020 à 09:41, James  a écrit :
> >
> > > Hi
> > >
> > > For the life of me, I can't find anything related to exporting
> templates
> > > to new systems/environments along with the parameters.
> > >
> > > I don't need the parameter values necessarily, but, importing a new
> > > template and then needing to create the parameters manually while
> setting
> > > them seems a little tedious. Is this only possible using a registry
> perhaps?
> > >
> > > Any guidance would be helpful.
> > >
> > > thanks
> > >
> >
>
> --
>
> Sent from Gmail Mobile
>
> --
>
> Sent from Gmail Mobile
>

RE: Creating parameters from exported template?

[nathan.english]

Hi Bryan,

Would the same also work by using two separate git persistence providers 
pointed at two different git repositories? We have a remote production instance 
with no remote access (effectively airgapped).

Our plan is when we deploy to production, provide a full exported copy of the 
git repository, and it’s history to a locally running instance of a git server 
in production. Which we will then configure the production NiFi Registry to use 
that export of the git repository for read-only operations.

So two separate Nifi Registries with two independent git repo’s, the production 
one as a cloned export of the Dev/Test Git Repo.

We are planning on testing this within the next few weeks.

Kind Regards,

Nathan

From: Bryan Bende [mailto:bbe...@gmail.com]
Sent: 19 May 2020 15:36
To: users@nifi.apache.org
Subject: Re: Creating parameters from exported template?

John,

You can have a nifi registry on each network and use Nifi CLI to export and 
import the flows between registries. There is one limitation in that nested 
versioning doesn’t currently work when doing that.

Also, the parameter contexts do live outside the flow at the controller level, 
however in order to use one of them, the use needs read access to the context 
and write access to the process group. So you can setup fine grained policies 
on each context to restrict who can use it.

Thanks,

Bryan

On Tue, May 19, 2020 at 8:26 AM Bryan Bende 
mailto:bbe...@gmail.com>> wrote:
The best option for moving flows across environments would be versioned flows 
in Nifi registry, which have full support for parameters.

On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin 
mailto:lapinoujou...@gmail.com>> wrote:
Hum, did not see that, and because I am still using "old" (1.9.2) version where 
parameters are not available.

But I understand like you

Le mar. 19 mai 2020 à 13:56, James 
mailto:genios...@gmail.com>> a écrit :
Hi

Thanks for the response.

According to this, it's not recommended: 
https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables

"Variables and the nifi.variable.registry.properties property will be removed 
in a future release. As a result, it is highly recommended to switch to 
Parameters."

Or am I thinking of the wrong thing?

thanks

On 2020/05/19 08:11:14, Etienne Jouvin 
mailto:lapinoujou...@gmail.com>> wrote:
> Hi.
>
> In this case why don't you use variables ?
>
> Etienne
>
>
> Le mar. 19 mai 2020 à 09:41, James 
> mailto:genios...@gmail.com>> a écrit :
>
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
>
--
Sent from Gmail Mobile
--
Sent from Gmail Mobile

Re: Creating parameters from exported template?

[Bryan Bende]

John,

You can have a nifi registry on each network and use Nifi CLI to export and
import the flows between registries. There is one limitation in that nested
versioning doesn’t currently work when doing that.

Also, the parameter contexts do live outside the flow at the controller
level, however in order to use one of them, the use needs read access to
the context and write access to the process group. So you can setup fine
grained policies on each context to restrict who can use it.

Thanks,

Bryan

On Tue, May 19, 2020 at 8:26 AM Bryan Bende  wrote:

> The best option for moving flows across environments would be versioned
> flows in Nifi registry, which have full support for parameters.
>
> On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin 
> wrote:
>
>> Hum, did not see that, and because I am still using "old" (1.9.2) version
>> where parameters are not available.
>>
>> But I understand like you
>>
>> Le mar. 19 mai 2020 à 13:56, James  a écrit :
>>
>>> Hi
>>>
>>> Thanks for the response.
>>>
>>> According to this, it's not recommended:
>>> https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables
>>>
>>> "Variables and the nifi.variable.registry.properties property will be
>>> removed in a future release. As a result, it is highly recommended to
>>> switch to Parameters."
>>>
>>> Or am I thinking of the wrong thing?
>>>
>>> thanks
>>>
>>> On 2020/05/19 08:11:14, Etienne Jouvin  wrote:
>>> > Hi.
>>> >
>>> > In this case why don't you use variables ?
>>> >
>>> > Etienne
>>> >
>>> >
>>> > Le mar. 19 mai 2020 à 09:41, James  a écrit :
>>> >
>>> > > Hi
>>> > >
>>> > > For the life of me, I can't find anything related to exporting
>>> templates
>>> > > to new systems/environments along with the parameters.
>>> > >
>>> > > I don't need the parameter values necessarily, but, importing a new
>>> > > template and then needing to create the parameters manually while
>>> setting
>>> > > them seems a little tedious. Is this only possible using a registry
>>> perhaps?
>>> > >
>>> > > Any guidance would be helpful.
>>> > >
>>> > > thanks
>>> > >
>>> >
>>>
>> --
> Sent from Gmail Mobile
>
-- 
Sent from Gmail Mobile

Re: Creating parameters from exported template?

[Bryan Bende]

The best option for moving flows across environments would be versioned
flows in Nifi registry, which have full support for parameters.

On Tue, May 19, 2020 at 8:01 AM Etienne Jouvin 
wrote:

> Hum, did not see that, and because I am still using "old" (1.9.2) version
> where parameters are not available.
>
> But I understand like you
>
> Le mar. 19 mai 2020 à 13:56, James  a écrit :
>
>> Hi
>>
>> Thanks for the response.
>>
>> According to this, it's not recommended:
>> https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables
>>
>> "Variables and the nifi.variable.registry.properties property will be
>> removed in a future release. As a result, it is highly recommended to
>> switch to Parameters."
>>
>> Or am I thinking of the wrong thing?
>>
>> thanks
>>
>> On 2020/05/19 08:11:14, Etienne Jouvin  wrote:
>> > Hi.
>> >
>> > In this case why don't you use variables ?
>> >
>> > Etienne
>> >
>> >
>> > Le mar. 19 mai 2020 à 09:41, James  a écrit :
>> >
>> > > Hi
>> > >
>> > > For the life of me, I can't find anything related to exporting
>> templates
>> > > to new systems/environments along with the parameters.
>> > >
>> > > I don't need the parameter values necessarily, but, importing a new
>> > > template and then needing to create the parameters manually while
>> setting
>> > > them seems a little tedious. Is this only possible using a registry
>> perhaps?
>> > >
>> > > Any guidance would be helpful.
>> > >
>> > > thanks
>> > >
>> >
>>
> --
Sent from Gmail Mobile

Re: Creating parameters from exported template?

[John McGinn]

 
Etienne,

I'm in the same boat as you. Currently using 1.9.2, and haven't upgraded yet, 
so only know of variables. I'm also concerned about migrating processor groups, 
as templates, from one environment to another, which cannot be done via NiFi 
Registry due to different networks/companies/firewalls. Currently, the 
variables are included as the part of the template, but unsure how the 
parameter context will be involved in the template. As well, those 
variables/parameters are now global, and not self-contained in a processor 
group or template, so any processor group can use any parameter context on a 
whim, rather than having to create/define it locally. I'm trying to plan out 
time for investigating the newer versions, and how parameter contexts will need 
to be used going forward.

John
 On Tuesday, May 19, 2020, 08:01:58 AM EDT, Etienne Jouvin 
 wrote:  
 
 Hum, did not see that, and because I am still using "old" (1.9.2) version 
where parameters are not available.

But I understand like you
Le mar. 19 mai 2020 à 13:56, James  a écrit :

Hi

Thanks for the response. 

According to this, it's not recommended: 
https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables

"Variables and the nifi.variable.registry.properties property will be removed 
in a future release. As a result, it is highly recommended to switch to 
Parameters."

Or am I thinking of the wrong thing?

thanks

On 2020/05/19 08:11:14, Etienne Jouvin  wrote: 
> Hi.
> 
> In this case why don't you use variables ?
> 
> Etienne
> 
> 
> Le mar. 19 mai 2020 à 09:41, James  a écrit :
> 
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
> 

  

Re: Creating parameters from exported template?

[Etienne Jouvin]

Hum, did not see that, and because I am still using "old" (1.9.2) version
where parameters are not available.

But I understand like you

Le mar. 19 mai 2020 à 13:56, James  a écrit :

> Hi
>
> Thanks for the response.
>
> According to this, it's not recommended:
> https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables
>
> "Variables and the nifi.variable.registry.properties property will be
> removed in a future release. As a result, it is highly recommended to
> switch to Parameters."
>
> Or am I thinking of the wrong thing?
>
> thanks
>
> On 2020/05/19 08:11:14, Etienne Jouvin  wrote:
> > Hi.
> >
> > In this case why don't you use variables ?
> >
> > Etienne
> >
> >
> > Le mar. 19 mai 2020 à 09:41, James  a écrit :
> >
> > > Hi
> > >
> > > For the life of me, I can't find anything related to exporting
> templates
> > > to new systems/environments along with the parameters.
> > >
> > > I don't need the parameter values necessarily, but, importing a new
> > > template and then needing to create the parameters manually while
> setting
> > > them seems a little tedious. Is this only possible using a registry
> perhaps?
> > >
> > > Any guidance would be helpful.
> > >
> > > thanks
> > >
> >
>

Re: Creating parameters from exported template?

[James]

Hi

Thanks for the response. 

According to this, it's not recommended: 
https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Variables

"Variables and the nifi.variable.registry.properties property will be removed 
in a future release. As a result, it is highly recommended to switch to 
Parameters."

Or am I thinking of the wrong thing?

thanks

On 2020/05/19 08:11:14, Etienne Jouvin  wrote: 
> Hi.
> 
> In this case why don't you use variables ?
> 
> Etienne
> 
> 
> Le mar. 19 mai 2020 à 09:41, James  a écrit :
> 
> > Hi
> >
> > For the life of me, I can't find anything related to exporting templates
> > to new systems/environments along with the parameters.
> >
> > I don't need the parameter values necessarily, but, importing a new
> > template and then needing to create the parameters manually while setting
> > them seems a little tedious. Is this only possible using a registry perhaps?
> >
> > Any guidance would be helpful.
> >
> > thanks
> >
> 

Re: Creating parameters from exported template?

[Etienne Jouvin]

Hi.

In this case why don't you use variables ?

Etienne


Le mar. 19 mai 2020 à 09:41, James  a écrit :

> Hi
>
> For the life of me, I can't find anything related to exporting templates
> to new systems/environments along with the parameters.
>
> I don't need the parameter values necessarily, but, importing a new
> template and then needing to create the parameters manually while setting
> them seems a little tedious. Is this only possible using a registry perhaps?
>
> Any guidance would be helpful.
>
> thanks
>

Creating parameters from exported template?

[James]

Hi

For the life of me, I can't find anything related to exporting templates to new 
systems/environments along with the parameters. 

I don't need the parameter values necessarily, but, importing a new template 
and then needing to create the parameters manually while setting them seems a 
little tedious. Is this only possible using a registry perhaps?

Any guidance would be helpful.

thanks