of NiFi I am running is 1.1.1
Below are my relevant config settings and the resultant log output which is
the point at which I am stuck. Any help is very much appreciated. As an
FYI, everywhere below that says "my.ip.address" is the actual IP address of
my AWS server.
Thanks in Advanc
blog post that shows
> how to do it using the toolkit in client-server mode:
> https://pierrevillard.com/2016/11/29/apache-nifi-1-1-0-
> secured-cluster-setup/
>
> Hope that helps.
>
> -Bryan
>
>
> On Wed, Mar 1, 2017 at 8:43 AM, Ryan H
> wrote:
> > Dear NiFi D
id that I would follow up with my full solution once
everything seems to be stable so any other members that may run in to the
issue have another documented solution.
Thanks for the help and direction! I will follow up in the next day with my
solution.
Cheers,
Ryan H.
On Wed, Mar 1, 2017 at 9:36 AM
?
Cheers,
Ryan H
server and use the
Hostname instead of the IP address. This corrected the problem and allowed
me to continue with the Secure Cluster Setup.
Thanks for the Help!!
Cheers,
Ryan H.
On Wed, Mar 1, 2017 at 1:11 PM, Ryan H
wrote:
> Hi Andy,
>
> Thanks you as well for a quick response.
>
>
Hi Joe,
Thanks for the quick response. I will follow that thread to see what the
outcome is.
Cheers,
Ryan H.
On Thu, Mar 2, 2017 at 3:36 PM, Joe Witt wrote:
> Ryan,
>
> We had a very similar question just a bit ago.
>
> https://lists.apache.org/thread.html/1289510c6d69176
. Setup multiple nodes Unsecured across multiple EC2 instances.
4. Setup single standalone Secured NiFi.
5. Setup multiple nodes Secured (clustered) on single EC2 instance.
Below are the relevant config files for my 2 nodes. Any help is greatly
appreciated!
Cheers,
Ryan H
ant or not.
Cheers,
Ryan H.
-
EC2 Instance 1
-
nifi.properties
nifi.state.management.embedded.zookeeper.start=true
# Site to Site properties
nifi.remote.input.host=my-host-name-1
nifi.remote.input.s
retry the cluster
once that has been resolved. I will follow up once that has been worked out
and whether or not that resolves the issue (makes sense that it would).
Cheers,
Ryan H.
On Tue, Mar 7, 2017 at 11:00 AM, Bryan Bende wrote:
> Hi Ryan,
>
> Is each instance able to success
on-and-multi-tenancy
https://pierrevillard.com/2016/11/29/apache-nifi-1-1-0-secured-cluster-setup/
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#security-configuration
Any help or direction is greatly appreciated!
Cheers,
Ryan H.
12:39:41,110 INFO [NiFi Web Server-117]
o.a.n.w.a.c.IllegalStateExceptionMapper
java.lang.IllegalStateException: Kerberos ticket login not supported by
this NiFi.. Returning Conflict response.
Any thoughts on this? Or is this possibly a misleading error shown in the
nifi-user.log...
-Ryan H.
:
no-referrer-when-downgrade
1. Request URL:
https://my-server-address/nifi-api/flow/current-user
2. Request Method:
GET
3. Status Code:
500 Internal Server Error
4. Remote Address:
10.227.80.39:443
5. Referrer Policy:
no-referrer-when-downgrade
Cheers,
Ryan H
o the errors mentioned
in that email.
Cheers,
Ryan H.
On Thu, Mar 16, 2017 at 1:21 PM, Bryan Bende wrote:
> Ryan,
>
> I haven't had time to look into your other email yet, but I can
> confirm that Kerberos is not required for a secure cluster, and you
> always end up see
.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Mar 16, 2017, at 6:04 AM, Ryan H
> wrote:
>
> Hi All,
>
> I am working on setting up a 2 node secure NiFi Cluster (version 1.1.1). I
> am using the NiFi Toolkit in standalone mode for the
Hi,
I'm curious if there's a way to get the ComponentLog to output to
StandardOut during a Unit Test.
I see I can access them when I call
testRunner.getLogger().getErrorMessages(), however, I'd really like to see
a stacktrace if it happens without having to iterate through an array to
find i
.xml file from one of the other modules (be sure to use one from the
> *test* directory, not the *main* directory).
>
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Sep 26, 20
, Ryan H
wrote:
> Hi Andy,
>It's a custom processor I'm writing. I was scratching my head
> wondering if that is it, or adding a test dependency for logback has
> something to do with it.
>
>I just copy/pasted this one: https:/
ep 27, 2017 at 11:08 AM, Bryan Bende wrote:
> I think another option is to use the simple SLF4J logger
>
>
> org.slf4j
> slf4j-simple
> test
>
>
> Then you probably wouldn't need the logback.xml file since you
> wouldn't be logging through
t scope, knowing the nifi container will have a copy of it.
Thanks,
Ryan
On Wed, Sep 27, 2017 at 12:50 PM, Ryan H
wrote:
> *tl;dr:* When using TestRunner, I can only get logging to log to the
> console/standard out when logback-classic is present in the test-scope.
>
> Is it
Hi,
I'm looking for a NiFi Processor Lifecycle Diagram to show when the
lifecycle events get kicked off, such as @OnPropertyModified, or
@Started/Stopped, etc.
I've read through the docs a few times, although it's been a while since
I've really scoured all the links, I can't seem to find one
che.org/docs/nifi-docs/html/developer-
> guide.html#component-lifecycle
> [2] https://nifi.apache.org/docs/nifi-docs/html/nifi-in-
> depth.html#life-of-a-flowfile
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BA
FYI - NiFi is now listed here:
https://raw.githubusercontent.com/cncf/landscape/master/landscape/CloudNativeLandscape_latest.jpg
https://github.com/cncf/landscape#current-version
Ryan
,
Ryan H.
://issues.apache.org/jira/browse/NIFI-3642 since jetty can't bind to a
VIP.
-Ryan H
On Mon, Jan 29, 2018 at 12:28 PM, Bryan Bende wrote:
> Ryan,
>
> I remember creating an issue for something that seems similar to what
> you are running into:
>
> https://issues.apache.org/jira/br
Good deal, thanks for getting back to me about it. I think that I may make
some changes and potentially open a PR depending on what comes of it (I'll
have to take a look at the workflow and all of that good stuff first).
Cheers,
Ryan H
On Mon, Jan 29, 2018 at 12:55 PM, Bryan Bende
address for other nodes in the cluster to use (in our case a configured
VIP) for the UI/rest api calls. Hope this makes sense.
Cheers,
Ryan H
On Tue, Jan 30, 2018 at 11:54 AM, Daniel Chaffelson
wrote:
> Hi Ryan,
> I have proposed a small change to the Docker image which may help you here
>
tise for me and would rather
be clear on what I'm doing.
Any help is greatly appreciated.
Cheers,
Ryan H
ey;*
...
}
Cheers,
Ryan H
On Wed, Feb 7, 2018 at 9:52 PM, Koji Kawamura
wrote:
> Hi Ryan,
>
> Although I am not sure why you'd want to use http between the clients
> and Nginx, I was able to setup similar environment.
> I used LDAP provider instead of OpenID, but OpenID should
; NGINX -> secure NiFi. The http
hop will soon move to be https, thus the interim solution until that is
available. I think this is the answer to my question for what Koji recently
replied with as a solution. I'll give it a go and follow up. Thanks for the
help thus far!
Cheers,
Ryan H
On Wed
e the error is being thrown from
''FileAccessPolicyProvider.populateNodes()")
Any help is always appreciated!
Cheers,
Ryan H
My Config:
*authorizers.xml*
file-user-group-provider
org.apache.nifi.authorization.FileUserGroupProvider
./conf/users.xml
Hi Kevin,
Thanks for the quick reply! That did the trick. Quick question, does the
order of the "Initial User Identities" matter as the "Initial Node
Identities" do? Or are only the Node Identities order important?
-Ryan H
On Sat, Feb 17, 2018 at 5:34 PM, Kevin Dora
st 1.2.3.4;
...
proxy_pass http://1.2.3.4:8080;
}
}
Hope this helps!
-Ryan H
On Fri, Feb 16, 2018 at 6:19 AM, Sean Marciniak wrote:
> Hey team,
>
> I have NiFi running on a standalone VM and I try to directly connect to it
> over http and I get this message:
>
usersGroupProvider list on
the secure NiFi cluster nodes.
Any thoughts?
Thanks,
Ryan H
Hi,
Yes, that looks like it is the issue. I think I have run into this problem
before using IP's instead of hostnames. I have it working now.
Thanks for the quick response!
-Ryan H
On Wed, Feb 21, 2018 at 10:15 AM, Bryan Bende wrote:
> Ryan,
>
> Did you happen to enter the regi
Hi Andy,
Yes, thanks for the suggestion. Ultimately that is what I want to do for
this specific situation. I just looked at the toolkit and saw that you are
able to add in SAN's. I am going to try that route.
Cheers,
Ryan H.
On Wed, Feb 21, 2018 at 10:38 AM, Andy LoPresto
wrote:
&
Hi All,
I am trying to set up a secure NiFi cluster (or just a single node to start
with rather) that uses Knox for AuthN. I want to configure Knox with an
OpenID provider. From what I can tell I have two options:
1. Access NiFi directly which would then kick back to Knox for Auth (which
is then c
7;ve created a JIRA [1] and will work on adding the
> documentation.
>
> [1] https://issues.apache.org/jira/browse/NIFI-4931
>
> On Sat, Mar 3, 2018 at 4:14 PM Ryan H
> wrote:
>
>> Hi All,
>>
>> I am trying to set up a secure NiFi cluster (or just a single node t
Hi All,
I have been working on getting a secure NiFi cluster to work with Knox. I
would like to have Knox be the entry point to NiFi. I have a NiFi cluster
running in secure mode without error. Now I would like to place Knox in
front of the Cluster. I have KnoxSSO setup which is configured with an
.
>
> After adding the keystore and truststore material to gateway.jks, added a
> user and policy for NiFi to identify and authorize Knox for proxying, and
> updated nifi.properties mentioned above, Knox should be able to proxy NiFi
> securely.
>
> On Thu, Mar 8, 2018 at 8:15 AM
sure if this is helpful or not, but just wanted to share what
we have done to work with what is available.
Cheers,
Ryan H
On Wed, Apr 4, 2018 at 4:07 PM, Mike Thomsen wrote:
> Aamir,
>
> I was looking for a way to do that w/ the Docker image without injecting a
> new bootstrap.c
+1 on this idea. We were considering doing this as well as we have ran into
issues with the system getting borked because of users unknowingly doing
the same with items such as GenerateFlowFile and some others. I think its a
great idea!
Cheers,
Ryan H
On Thu, Apr 26, 2018 at 8:14 AM, Pierre
Hi,
Are there updated NiFi docs for upgrading NiFi's that have the new
access controls? eg. 1.3 -> 1.4 or 1.4 -> 1.6? There looks like a couple
more files that need to be moved over... users.xml, authorizations.xml.. I
copied over the state folder (although I'm not sure if I had to)
I'm lo
ation-
> guide.html#multi-tenant-authorization
>
> Andy LoPresto
> alopre...@apache.org
> *alopresto.apa...@gmail.com *
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On May 2, 2018, at 11:48 AM, Ryan H wrote:
>
> Hi,
>Are there updated N
Hi All,
We have found what seems to be an issue when creating a template of a
Process Group that is under version control with NiFi Registry. What seems
to happen is that there is a hard reference to the Registry Instance from
the Env that the template was created from within the template xml file
apache-nifi-with-the-nifi-registry/
>
> [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/
> nifi-toolkit-cli
>
> [3] https://nifi.apache.org/download.html
>
> [4] https://github.com/Chaffelson/nipyapi
>
> [5] https://community.hortonworks.com/articles/177301/big-dat
version control. Templates were created with no
> > registry in sight, and, frankly, should not be used anymore now.
> >
> > If registries are setup, use the tools meant for the registry (and come
> for
> > any help here).
> >
> > Cheers!
> > Andre
is definitely one where we could
> improve the documentation, and possibly even NiFi’s handling of the
> Registry metadata when exporting or importing templates if it is not a huge
> effort.
>
>
>
> Thanks,
>
> Kevin
>
>
>
> *From: *Ryan H
> *Reply-To: *
>
; >> documented, and, ideally, migration tools and guides should be provided.
> >> This case you’ve encountered is definitely one where we could improve
> the
> >> documentation, and possibly even NiFi’s handling of the Registry
> metadata
> >> when exporting
0,
"output": "value",
"flowFilesTransferred": 0,
"bytesTransferred": 0,
"transferred": "value",
"bytesReceived": 0,// I think this is the one, but not sure
"flowFilesReceived": 0,
"received": "value",
"bytesSent": 0, // I think this is the other one, but not sure
"flowFilesSent": 0,
"sent": "value",
"activeThreadCount": 0,
"terminatedThreadCount": 0
},
"nodeSnapshots": [{…}]
},
"canRead": true
}
Any help or insight is always appreciated!
Cheers,
Ryan H.
wish NiFi provided nicer out of the box logging/monitoring capabilities
> but on a bright side, it seems to me that you can build your own and
> customize it as you want.
>
>
> On Tue, Jul 24, 2018 at 10:55 PM Ryan H
> wrote:
>
>> Hi All,
>>
>> I am lookin
uot;,
"flowFilesTransferred": 0,
"bytesTransferred": 0,
"transferred": "value",
* "bytesReceived": 0,// I think this is the one, but not sure*
"flowFilesReceived": 0,
*"received": "value",*
xt snapshot at time t = 8 hours, you'll only
> have 1 hour worth of data from the node that restarted... this is one of
> the benefits of gleaning this info from Provenance data.
>
> Alternatively, I suppose, some sort of persistent reporting mechanism
> could be built within Ni
es not need to be a Jolt spec and can be modified to meet the needs of
whatever would work for this.
I started to look into schema registry, but kind of got lost a bit and
wasn't sure if it could be applied to this situation.
Any Help is, as always, greatly appreciated.
Cheers,
Ryan H.
Full_Name
I should mention that System-A is the only modifiable schema; System-B is
fixed. The only thing that can change is the System-A schema and its
mapping to System-B and will be unique to each Organization.
Hope this clarifies a bit more.
Cheers,
Ryan H
On Thu, Oct 4, 2018 at 2:39
, whatever that looks like for the given environment.
Hope this helps,
Ryan H.
On Wed, Jan 16, 2019 at 12:37 PM Erik Anderson wrote:
> Ok,
>
> I guess the better question to as is
>
> If NiFi is running in an ephemeral environment or mode such as kubernetes
> and docker with
t really find
much in way of documentation for this specific case and just want to
understand what options there are without breaking any of the clusters and
get some security in there.
Any info is always appreciated!
Cheers,
Ryan H
efore, but
now it seems that it is. I didn't see this as a known issue in any of the
migration guides or release notes. Can anyone provide any insight on this?
Steps to remediate? Is the cause exactly what the error message says,
having a blank value for this config prop?
Cheers,
Ryan H
h will take time. It requires improved tooling out of
> the box and we want the 'initial up and running experience' to be as smooth
> as possible and similar to how it works now.
>
> Anyway, hopefully that helps explain.
>
> Thanks
> Joe
>
> On Thu, Jan 24, 2019 a
:33:08,479 ERROR [NiFi logging handler] org.apache.nifi.StdErr
Shutting down...
-Ryan H
On Thu, Jan 24, 2019 at 11:48 AM Joe Witt wrote:
> ryan
>
> right...i think you are facing something else. We have locked down on
> some headers, etc.. They are mentioned in the migration guide.
I thought it was. Thanks for the quick response and help!
Cheers,
Ryan H
On Thu, Jan 24, 2019 at 12:41 PM Joe Witt wrote:
> Ryan,
>
> Got ya ... so bootstrap log seems to indicate badness. Can you please
> share more of the nifi-app.log as there will almost certainly be a smok
same we got it worked out. I was just a bit confused that the above
came thru as a WARN instead of an ERROR.
Cheers,
Ryan H
On Thu, Jan 24, 2019 at 12:54 PM Andy LoPresto wrote:
> Ryan,
>
> I added that warning [1], and Joe’s right, it should have been at a WARN
> severity. However, i
even in these batches which make
it manageable but still very slow.
Thanks in advance!
Cheers,
Ryan H
ed to throw the question
out to the community first before delving into this.
Cheers,
Ryan H
Process Group level (not sure if there is a good case for it
other than this scenario)?
-Ryan H
On Wed, Mar 27, 2019 at 2:07 PM Andy LoPresto wrote:
> Ryan,
>
> Reading through your email, my immediate suggestion was NiPyAPI. I think
> Dan has wrapped some useful query methods ther
Sounds like the ability to disable at the Process Group exists after 1.7, I
retract my question!
-Ryan H
On Wed, Mar 27, 2019 at 2:18 PM Ryan H
wrote:
> @Mark, we are on version 1.5 and there is a definite slow down, at least
> for us with UI responsiveness (initial load of canvas upon l
in cases like this?
Is there a way for us to make NiFi more resilient to flow misconfigurations
that cause it to crash?
As always, any thoughts or input is greatly appreciated.
Cheers,
Ryan H.
Hi Bryan,
Thanks for this. I don't remember seeing this setting, but going to check
it out.
-Ryan
On Thu, May 2, 2019 at 12:43 PM Bryan Bende wrote:
> You can set nifi.flowcontroller.autoResumeState=false to start NiFi
> without running everything.
>
> On Thu, May 2, 2019 a
rs to only access certain buckets may be important.
For now just being able to get this hooked up is ideal. Thoughts?
Thanks in Advance,
Ryan H.
user. I hope I summarized this correctly.
As always, thanks for the quick responses and help.
Cheers,
Ryan H
On Thu, Oct 24, 2019 at 8:33 AM Kevin Doran wrote:
> This is a very good question, and Pierre gives a good summary of how
> to go about solving for it.
>
> Essentially, you
69 matches
Mail list logo
