Thanks, Joe.
Our use case is getting data from a source and ingest it into a
kerberized hive. We do it with a PutHive3QL processor, which uses a
Hive3ConnectionPool
controller service, which uses a KeytabCredentialsService controller
service. I'm not pretty sure about what crossrealm is, so I guess we don't
use it. We authenticate against the kerberos server where our principal is
stored.
We are going to 292 because of the nifis requirement of being in 251 or
later, and not being the last. But we have tested with 311 with the same
result. I didnt hear about Azul dist, we will take a look and let you know.
This is the error log:
2022-01-27 17:53:27,463 ERROR [Timer-Driven Process Thread-14]
o.a.n.c.s.StandardControllerServiceNode
StandardControllerServiceNode[service=Hive3ConnectionPool[id=356efabb-5e9d-394c-a719-86b6b65ad2e8],
versionedComponentId=null,
processGroup=StandardProcessGroup[identifier=78f004f1-f873-3a33-855a-553e0a114b68,name=RADAR_DONE],
active=true] Failed to invoke @OnEnabled method due to
org.apache.nifi.reporting.InitializationException:
org.apache.nifi.util.hive.AuthenticationFailedException: Kerberos
Authentication for Hive failed: {}
org.apache.nifi.reporting.InitializationException:
org.apache.nifi.util.hive.AuthenticationFailedException: Kerberos
Authentication for Hive failed
at
org.apache.nifi.dbcp.hive.Hive3ConnectionPool.onConfigured(Hive3ConnectionPool.java:435)
at sun.reflect.GeneratedMethodAccessor266.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:142)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:130)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:75)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:52)
at
org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:432)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.nifi.util.hive.AuthenticationFailedException:
Kerberos Authentication for Hive failed
at
org.apache.nifi.util.hive.HiveConfigurator.authenticate(HiveConfigurator.java:94)
at
org.apache.nifi.dbcp.hive.Hive3ConnectionPool.onConfigured(Hive3ConnectionPool.java:432)
... 16 common frames omitted
Caused by: java.io.IOException: Unable to acquire UGI for KerberosUser:
Unable to login with ******************* due to: Message stream modified
(41)
at
org.apache.nifi.hadoop.SecurityUtil.getUgiForKerberosUser(SecurityUtil.java:109)
at
org.apache.nifi.util.hive.HiveConfigurator.authenticate(HiveConfigurator.java:92)
... 17 common frames omitted
Caused by: javax.security.auth.login.LoginException: Unable to login with
******************* due to: Message stream modified (41)
at
com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:808)
at
com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618)
at sun.reflect.GeneratedMethodAccessor226.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at
javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at
javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at
javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at
javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at
org.apache.nifi.security.krb.AbstractKerberosUser.login(AbstractKerberosUser.java:81)
at
org.apache.nifi.hadoop.SecurityUtil.getUgiForKerberosUser(SecurityUtil.java:96)
... 18 common frames omitted
Thanks again.
Regards.
Guille
El jue, 27 ene 2022 a las 17:01, Joe Witt (<[email protected]>) escribió:
> Guille
>
> We are trying to be broadly compatible with every Java 8 and Java 11
> release we can but of course the older they get certain ones become
> unusable and the newer they get sometimes new behavior are introduced.
> We test a lot of combinations directly plus we hear a lot from threads
> like this. But we simply can't know all/verify combinations. So
> generally the answer is 'it should work' but of course sometimes
> pieces break as the JVM changes.
>
> In this case you'll need to tell us more about your configuration for
> us to really try/consider much. We'd need to hear about how you use
> Kerb(do you use cross realm?) and we'd need to see the actual error's
> you're seeing. Also why go to 282 now if there are much newer
> versions available? I'm not sure about openjdk and its supported
> status in Java 8. But you might want to also look at Azul or other
> JDK providers.
>
> Thanks
>
> On Thu, Jan 27, 2022 at 8:40 AM Guillermo Muñoz Salgado
> <[email protected]> wrote:
> >
> > Hi all,
> >
> > We are upgrading Java from OpenJDK1.8.222 to OpenJDK1.8.292, and
> everything seems to be ok except the Kerberos Controller Services. We think
> this issue [1] can be related. To mitigate it we launch NiFi with the next
> property in the bootstrap.conf file:
> java.arg.17=-Dsun.security.krb5.disableReferrals=true, but we get the same
> results.
> >
> > Are Kerberos Controller Services compatible with OpenJDK 1.8.282?
> > Anyone else with similar problems out there?
> >
> > I paste our bootstrap.conf:
> >
> >
> > java=java
> >
> > preserve.environment=false
> >
> > lib.dir=./lib
> >
> > conf.dir=./conf
> >
> > graceful.shutdown.seconds=20
> >
> > java.arg.1=-Dorg.apache.jasper.compiler.disablejsr199=true
> >
> > java.arg.2=-Xms4G
> >
> > java.arg.3=-Xmx8G
> >
> > java.arg.4=-Djava.net.preferIPv4Stack=true
> >
> > java.arg.5=-Dsun.net.http.allowRestrictedHeaders=true
> >
> > java.arg.6=-Djava.protocol.handler.pkgs=sun.net.www.protocol
> >
> > java.arg.13=-XX:+UseG1GC
> >
> > java.arg.14=-Djava.awt.headless=true
> >
> > nifi.bootstrap.sensitive.key=
> >
> > java.arg.15=-Djava.security.egd=file:/dev/urandom
> >
> > java.arg.16=-Djavax.security.auth.useSubjectCredsOnly=true
> >
> > java.arg.17=-Dsun.security.krb5.disableReferrals=true
> >
> > java.arg.18=-Dzookeeper.admin.enableServer=true
> >
> > notification.services.file=./conf/bootstrap-notification-services.xml
> >
> > notification.max.attempts=5
> >
> >
> java.arg.curator.supress.excessive.logs=-Dcurator-log-only-first-connection-issue-as-error-level=true
> >
> >
> > Thanks in advance,
> > --
> > Guille
> >
> > [1] https://bugs.openjdk.java.net/browse/JDK-8233512
>
--
Guille
