Re: Provenance issues
Got ya. Would love to hear what you find when you give 1.13.2 a go thanks On Wed, Mar 31, 2021 at 7:41 PM Chad Zobrisky wrote: > We are on 1.12.1. Been trying to wait to update until 1.13 has been vetted > a little more since this is a prod system, unless the consensus is to go > straight to 1.13.2! > > Chad > > On Wed, Mar 31, 2021 at 10:36 PM Joe Witt wrote: > > > i *think* this is fixed in 1.13. What version are you on? > > > > On Wed, Mar 31, 2021 at 7:34 PM Chad Zobrisky > wrote: > > > > > Hello, > > > > > > We have a single nifi instance that is experiencing weird provenance > > > results. > > > > > > When querying for specific processors, there is no data returned and in > > the > > > logs we see... > > > > > > 2021-04-01 02:29:37,639 INFO [Provenance Query-1] > > > o.a.nifi.provenance.StandardQueryResult Completed Query[ > > > [e286d901-a721-4743-be19-5aed93d9582f] ] comprise > > > d of 1 steps in 5 millis. Index found 2 hits. Read 0 events from Event > > > Files. > > > there > > > > > > Is there any way to recover the events? New events for that processor > > also > > > aren't generated or queryable from the provenance UI; The index hits go > > up, > > > but no events returned still. > > > > > > Thanks, > > > Chad > > > > > >
Re: Provenance issues
We are on 1.12.1. Been trying to wait to update until 1.13 has been vetted a little more since this is a prod system, unless the consensus is to go straight to 1.13.2! Chad On Wed, Mar 31, 2021 at 10:36 PM Joe Witt wrote: > i *think* this is fixed in 1.13. What version are you on? > > On Wed, Mar 31, 2021 at 7:34 PM Chad Zobrisky wrote: > > > Hello, > > > > We have a single nifi instance that is experiencing weird provenance > > results. > > > > When querying for specific processors, there is no data returned and in > the > > logs we see... > > > > 2021-04-01 02:29:37,639 INFO [Provenance Query-1] > > o.a.nifi.provenance.StandardQueryResult Completed Query[ > > [e286d901-a721-4743-be19-5aed93d9582f] ] comprise > > d of 1 steps in 5 millis. Index found 2 hits. Read 0 events from Event > > Files. > > there > > > > Is there any way to recover the events? New events for that processor > also > > aren't generated or queryable from the provenance UI; The index hits go > up, > > but no events returned still. > > > > Thanks, > > Chad > > >
Re: Provenance issues
i *think* this is fixed in 1.13. What version are you on? On Wed, Mar 31, 2021 at 7:34 PM Chad Zobrisky wrote: > Hello, > > We have a single nifi instance that is experiencing weird provenance > results. > > When querying for specific processors, there is no data returned and in the > logs we see... > > 2021-04-01 02:29:37,639 INFO [Provenance Query-1] > o.a.nifi.provenance.StandardQueryResult Completed Query[ > [e286d901-a721-4743-be19-5aed93d9582f] ] comprise > d of 1 steps in 5 millis. Index found 2 hits. Read 0 events from Event > Files. > there > > Is there any way to recover the events? New events for that processor also > aren't generated or queryable from the provenance UI; The index hits go up, > but no events returned still. > > Thanks, > Chad >
Provenance issues
Hello, We have a single nifi instance that is experiencing weird provenance results. When querying for specific processors, there is no data returned and in the logs we see... 2021-04-01 02:29:37,639 INFO [Provenance Query-1] o.a.nifi.provenance.StandardQueryResult Completed Query[ [e286d901-a721-4743-be19-5aed93d9582f] ] comprise d of 1 steps in 5 millis. Index found 2 hits. Read 0 events from Event Files. there Is there any way to recover the events? New events for that processor also aren't generated or queryable from the provenance UI; The index hits go up, but no events returned still. Thanks, Chad
Re: Nifi authentication through Kerberos issues
It doesn't look like anything to me, but here's the stacktrace for when logback.xml has all of the user_file stuff in debug mode: 2021-03-31 22:54:13,670 INFO [NiFi Web Server-22] o.a.n.w.a.c.IllegalArgumentExceptionMapper java.lang.IllegalArgumentException: The supplied username and password are not valid.. Returning Bad Request response. 2021-03-31 22:54:13,672 DEBUG [NiFi Web Server-22] o.a.n.w.a.c.IllegalArgumentExceptionMapper java.lang.IllegalArgumentException: The supplied username and password are not valid. at org.apache.nifi.web.api.AccessResource.createAccessToken(AccessResource.java:734) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655) at org.apache.nifi.web.filter.RequestLogger.doFilter(RequestLogger.java:66) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:208) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:51) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.nifi.web.filter.ExceptionFilter.doFilter(ExceptionFilter.java:46) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634) at org.apache.nifi.web.security.ContentSecurityPolicyFilter.doFilter(ContentSecurityPolicyFilter.java:47) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.apache.nifi.web.server.JettyServer$2.doFilter(JettyServer.java:1048) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at
Re: Nifi authentication through Kerberos issues
Correct.
# kinit admin@MY.REALM
Password for admin@MY.REALM:
# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: admin@MY.REALM
Valid starting Expires Service principal
03/31/2021 22:42:10 04/01/2021 22:42:10 krbtgt/MY.REALM@MY.REALM
On Wed, Mar 31, 2021, 1:13 PM Bryan Bende wrote:
> So from a terminal on the nifi server, you can run "kinit
> admin@MY.REALM" and enter the password and it works, and this same
> principal and password entered into NiFi's login screen does not work?
>
> On Wed, Mar 31, 2021 at 2:19 PM Derek Richardson wrote:
> >
> > I'm working on transitioning a nifi instance we deploy with Kerberos and
> > I'm having some trouble authenticating. Everything looks correct, but
> when
> > I try to log in with any of my created users, I get an error message:
> >
> > The supplied username and password are not valid.
> >
> > Everything on nifi without https was working, and everything I've created
> > on the Kerberos side looks and works as expected, I just haven't been
> able
> > to get a user to log in to the Nifi UI.
> >
> > Here are some of my config files, is there anything I'm missing or have
> > incorrect?
> >
> > ---
> >
> > Authorizers.xml:
> >
> >
> >
> > file-user-group-provider
> >
> org.apache.nifi.authorization.FileUserGroupProvider
> > ./conf/users.xml
> >
> >
> >
> >
> >
> >
> > file-access-policy-provider
> >
> > org.apache.nifi.authorization.FileAccessPolicyProvider
> > file-user-group-provider
> > ./conf/authorizations.xml
> > admin@MY.REALM
>
> >
> >
> >
> >
> >
> >
> > managed-authorizer
> >
> > org.apache.nifi.authorization.StandardManagedAuthorizer
> > file-access-policy-provider
> >
> >
> >
> > file-provider
> > org.apache.nifi.authorization.FileAuthorizer
> > ./conf/authorizations.xml
> > ./conf/users.xml
> > admin@MY.REALM
>
> >
> >
> >
> >
> >
> >
> > -
> >
> > Relevant nifi.properties:
> > nifi.security.user.authorizer=file-provider
> > nifi.security.user.login.identity.provider=kerberos-provider
> > # kerberos #
> > nifi.kerberos.krb5.file= /etc/krb5.conf
> > nifi.kerberos.service.principal=admin@MY.REALM
> > nifi.kerberos.service.keytab.location=/etc/kadm5.keytab
> >
> > -
> >
> > Login-identity-provider.xml
> >
> >
> > kerberos-provider
> > org.apache.nifi.kerberos.KerberosProvider
> > MY.REALM
> > 12 hours
> >
> >
> >
> > ---
> >
> > /etc/krb5.conf:
> > [logging]
> > default = FILE:/var/log/krb5libs.log
> > kdc = FILE:/var/log/krb5kdc.log
> > admin_server = FILE:/var/log/kadmind.log
> >
> > [libdefaults]
> > ticket_lifetime = 24h
> > renew_lifetime = 7d
> > forwardable = true
> > default_realm = MY.REALM
> >
> > [realms]
> > RO.INTERNAL = {
> > kdc = nifi-djr5.ro.internal:88
> > admin_server = nifi-djr5.my.realm:749
> > default_domain = my.realm
> > }
> >
> > [domain_realm]
> > .my.realm = MY.REALM
> > my.realm = MY.REALM
> >
> > [kdc]
> > profile = /var/kerberos/krb5kdc/kdc.conf
> >
> > ---
> >
> > Any help would be greatly appreciated!
>
Re: [ANNOUNCE] New NiFi PMC Member Joey Frazee
Congrats joey! On Thu, Mar 25, 2021 at 3:44 PM Nathan Gough wrote: > > Congratulations Joey! Thanks for your past and future contributions. > > On Thu, Mar 25, 2021 at 3:15 PM Joey Frazee > wrote: > > > I want to make sure to say thanks for the recognition! I’m thrilled to be > > a part of this. The help and collaboration from the project members and > > contributors over the past years has made this a ton of fun and I’m very > > happy to be continuing with more of it. > > > > -joey > > > > > On Mar 25, 2021, at 11:57 AM, Otto Fowler > > wrote: > > > > > > Congratulations! > > > > > >> On Thu, Mar 25, 2021 at 2:54 PM Joe Witt wrote: > > >> > > >> NiFi Community, > > >> > > >> On behalf of the Apache NiFi PMC, I am pleased to announce that Joey > > Frazee > > >> has accepted the PMC's invitation to join the Apache NiFi PMC. > > >> > > >> Joey has been a contributor and committer of Apache NiFi for several > > >> years and has been involved across the whole spectrum of code reviews, > > >> JIRAs, code contributions, maintained a great 'awesome nifi' github > > >> site, and more. > > >> > > >> Please join us in congratulating and welcoming Joey to the Apache NiFi > > PMC. > > >> > > >> Congratulations Joey! > > >> > >
Re: Nifi authentication through Kerberos issues
So from a terminal on the nifi server, you can run "kinit
admin@MY.REALM" and enter the password and it works, and this same
principal and password entered into NiFi's login screen does not work?
On Wed, Mar 31, 2021 at 2:19 PM Derek Richardson wrote:
>
> I'm working on transitioning a nifi instance we deploy with Kerberos and
> I'm having some trouble authenticating. Everything looks correct, but when
> I try to log in with any of my created users, I get an error message:
>
> The supplied username and password are not valid.
>
> Everything on nifi without https was working, and everything I've created
> on the Kerberos side looks and works as expected, I just haven't been able
> to get a user to log in to the Nifi UI.
>
> Here are some of my config files, is there anything I'm missing or have
> incorrect?
>
> ---
>
> Authorizers.xml:
>
>
>
> file-user-group-provider
> org.apache.nifi.authorization.FileUserGroupProvider
> ./conf/users.xml
>
>
>
>
>
>
> file-access-policy-provider
>
> org.apache.nifi.authorization.FileAccessPolicyProvider
> file-user-group-provider
> ./conf/authorizations.xml
> admin@MY.REALM
>
>
>
>
>
>
> managed-authorizer
>
> org.apache.nifi.authorization.StandardManagedAuthorizer
> file-access-policy-provider
>
>
>
> file-provider
> org.apache.nifi.authorization.FileAuthorizer
> ./conf/authorizations.xml
> ./conf/users.xml
> admin@MY.REALM
>
>
>
>
>
>
> -
>
> Relevant nifi.properties:
> nifi.security.user.authorizer=file-provider
> nifi.security.user.login.identity.provider=kerberos-provider
> # kerberos #
> nifi.kerberos.krb5.file= /etc/krb5.conf
> nifi.kerberos.service.principal=admin@MY.REALM
> nifi.kerberos.service.keytab.location=/etc/kadm5.keytab
>
> -
>
> Login-identity-provider.xml
>
>
> kerberos-provider
> org.apache.nifi.kerberos.KerberosProvider
> MY.REALM
> 12 hours
>
>
>
> ---
>
> /etc/krb5.conf:
> [logging]
> default = FILE:/var/log/krb5libs.log
> kdc = FILE:/var/log/krb5kdc.log
> admin_server = FILE:/var/log/kadmind.log
>
> [libdefaults]
> ticket_lifetime = 24h
> renew_lifetime = 7d
> forwardable = true
> default_realm = MY.REALM
>
> [realms]
> RO.INTERNAL = {
> kdc = nifi-djr5.ro.internal:88
> admin_server = nifi-djr5.my.realm:749
> default_domain = my.realm
> }
>
> [domain_realm]
> .my.realm = MY.REALM
> my.realm = MY.REALM
>
> [kdc]
> profile = /var/kerberos/krb5kdc/kdc.conf
>
> ---
>
> Any help would be greatly appreciated!
Nifi authentication through Kerberos issues
I'm working on transitioning a nifi instance we deploy with Kerberos and
I'm having some trouble authenticating. Everything looks correct, but when
I try to log in with any of my created users, I get an error message:
The supplied username and password are not valid.
Everything on nifi without https was working, and everything I've created
on the Kerberos side looks and works as expected, I just haven't been able
to get a user to log in to the Nifi UI.
Here are some of my config files, is there anything I'm missing or have
incorrect?
---
Authorizers.xml:
file-user-group-provider
org.apache.nifi.authorization.FileUserGroupProvider
./conf/users.xml
file-access-policy-provider
org.apache.nifi.authorization.FileAccessPolicyProvider
file-user-group-provider
./conf/authorizations.xml
admin@MY.REALM
managed-authorizer
org.apache.nifi.authorization.StandardManagedAuthorizer
file-access-policy-provider
file-provider
org.apache.nifi.authorization.FileAuthorizer
./conf/authorizations.xml
./conf/users.xml
admin@MY.REALM
-
Relevant nifi.properties:
nifi.security.user.authorizer=file-provider
nifi.security.user.login.identity.provider=kerberos-provider
# kerberos #
nifi.kerberos.krb5.file= /etc/krb5.conf
nifi.kerberos.service.principal=admin@MY.REALM
nifi.kerberos.service.keytab.location=/etc/kadm5.keytab
-
Login-identity-provider.xml
kerberos-provider
org.apache.nifi.kerberos.KerberosProvider
MY.REALM
12 hours
---
/etc/krb5.conf:
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
default_realm = MY.REALM
[realms]
RO.INTERNAL = {
kdc = nifi-djr5.ro.internal:88
admin_server = nifi-djr5.my.realm:749
default_domain = my.realm
}
[domain_realm]
.my.realm = MY.REALM
my.realm = MY.REALM
[kdc]
profile = /var/kerberos/krb5kdc/kdc.conf
---
Any help would be greatly appreciated!
Re: Call for Presentations for ApacheCon 2021 now open
Hi all, just dripping in with a gentle reminder of the CFP and notifying you that this year's ApacheCon Asia also features the Integration track, so you might want to consider submitting your presentation there as well. I hope that we can have more software integration projects at ASF presented/promoted besides Camel on the Integration track. You can find links to the CFPs on Camel's website here: https://camel.apache.org/blog/2021/03/ApacheCons2021/ zoran On Tue, Mar 9, 2021 at 11:18 AM Zoran Regvart wrote: > > Hi folk, > On the backs of the resounding success from last year's, > ApacheCon@Home will be online, and again, we'll be featuring the > (Software) Integration track. I'm the chair of the Integration track > at ApacheCon@Home. And I'd like to invite folk from the Apache NiFi > community to participate by submitting a presentation for ApacheCon. > > Software integration is a broad topic and one I think Apache NiFi > folk can contribute considerably to the discussion. For the > Integration track, we wish to focus on showcasing the projects at ASF > that deal with software integration, and along with that, we also have > a preference for use case presentations. > > You can submit now till 1st of May at: > > https://acah2021.jamhosted.net/ > > Feel free to reach out to me if you have any questions, > > zoran > > -- Forwarded message - > From: Rich Bowen > Date: Mon, Mar 8, 2021 at 2:52 PM > Subject: Call for Presentations for ApacheCon 2021 now open > To: > > > The ApacheCon Planners and the Apache Software Foundation are pleased to > announce that ApacheCon@Home will be held online, September 21-23, 2021. > Once again, we’ll be featuring content from dozens of our projects, as > well as content about our community, how Apache works, business models > around Apache software, the legal aspects of open source, and many other > topics. > > Last year’s virtual ApacheCon@Home event was a big success, with 5,745 > registrants from more than 150 countries, spanning every time zone, with > the virtual format delivering content to attendees who would never have > attended an in-person ApacheCon (83% of post-event poll responders in > 2020 indicated this was their first ApacheCon ever)! > > Given the great participation and excitement for last year’s event, we > are announcing the Call for Presentations is now open to presenters from > around the world until May 1st. Talks can be focused on the topics > above, as well as any of our amazing projects. Submit your talks today! > > https://acah2021.jamhosted.net/ > > We look forward to reviewing your contribution to one of the most > popular open source software events in the world! > > > -- > Rich Bowen, VP Conferences > The Apache Software Foundation > https://apachecon.com/ > @apachecon > > > -- > Zoran Regvart -- Zoran Regvart
NiFi codebase warnings
Hi Devs/Maintainers I've recently started using NiFi and I thought it would be a good idea to give something back to the community while getting familiar with the different areas of NiFi. I am writing to explain the relation between few PRs that are submitted and check with you if it's a good idea and if it would require a different approach. I saw during compilation that are some warnings related to deprecated APIS, that could be solved without a lot of issues, so I've submitted a PR for each of the different APIs being deprecated, some of them internal, some of them external, [1],[2],[3],[4]. I've also submitted an small PR regarding code inspections [5] (Make inner classes static where possible). I guess after some more PRs all warnings would be gone and it would be possible to enable warnings as error during compilation. So those are the questions that popped in my mind: - Are these kind of initiatives wellcome? - Should I handle it differently at code level? - Should I do something different at Jira rather than creating individual issues? (Maybe an epic level that comprises all of this?) - Any particular improvement that sounds more urgent or you'd like to get it done first? Kind Regards JL [1] https://github.com/apache/nifi/pull/4947 [2] https://github.com/apache/nifi/pull/4945 [3] https://github.com/apache/nifi/pull/4942 [4] https://github.com/apache/nifi/pull/4941 [5] https://github.com/apache/nifi/pull/4940
