A couple of posts related to that error mention a solution related to removing the renew_liftetime from krb5.conf, or alternatively modifying the principal to allow renewals:
https://stackoverflow.com/questions/21001950/krbexception-message-stream-modified-41-when-connecting-to-smb-share-using-k Specifically the last answer there mentions making it work without removing the renew from kr5b.conf. Another one with similar solution: https://support.datastax.com/s/article/Attempts-to-connect-to-a-Kerberosenabled-cluster-running-on-CentOS-7-fails-with-LoginException-Message-stream-modified-41 On Thu, Jan 27, 2022 at 1:05 PM Joe Witt <[email protected]> wrote: > > And if you do not add this new system property what happens for you? > > On Thu, Jan 27, 2022 at 10:28 AM Guillermo Muñoz > <[email protected]> wrote: > > > > Thanks, Joe. > > > > Our use case is getting data from a source and ingest it into a kerberized > > hive. We do it with a PutHive3QL processor, which uses a > > Hive3ConnectionPool controller service, which uses a > > KeytabCredentialsService controller service. I'm not pretty sure about what > > crossrealm is, so I guess we don't use it. We authenticate against the > > kerberos server where our principal is stored. > > We are going to 292 because of the nifis requirement of being in 251 or > > later, and not being the last. But we have tested with 311 with the same > > result. I didnt hear about Azul dist, we will take a look and let you know. > > > > This is the error log: > > 2022-01-27 17:53:27,463 ERROR [Timer-Driven Process Thread-14] > > o.a.n.c.s.StandardControllerServiceNode > > StandardControllerServiceNode[service=Hive3ConnectionPool[id=356efabb-5e9d-394c-a719-86b6b65ad2e8], > > versionedComponentId=null, > > processGroup=StandardProcessGroup[identifier=78f004f1-f873-3a33-855a-553e0a114b68,name=RADAR_DONE], > > active=true] Failed to invoke @OnEnabled method due to > > org.apache.nifi.reporting.InitializationException: > > org.apache.nifi.util.hive.AuthenticationFailedException: Kerberos > > Authentication for Hive failed: {} > > org.apache.nifi.reporting.InitializationException: > > org.apache.nifi.util.hive.AuthenticationFailedException: Kerberos > > Authentication for Hive failed > > at > > org.apache.nifi.dbcp.hive.Hive3ConnectionPool.onConfigured(Hive3ConnectionPool.java:435) > > at sun.reflect.GeneratedMethodAccessor266.invoke(Unknown Source) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:142) > > at > > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:130) > > at > > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:75) > > at > > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:52) > > at > > org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:432) > > at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) > > at > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > > at > > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > > Caused by: org.apache.nifi.util.hive.AuthenticationFailedException: > > Kerberos Authentication for Hive failed > > at > > org.apache.nifi.util.hive.HiveConfigurator.authenticate(HiveConfigurator.java:94) > > at > > org.apache.nifi.dbcp.hive.Hive3ConnectionPool.onConfigured(Hive3ConnectionPool.java:432) > > ... 16 common frames omitted > > Caused by: java.io.IOException: Unable to acquire UGI for KerberosUser: > > Unable to login with ******************* due to: Message stream modified > > (41) > > at > > org.apache.nifi.hadoop.SecurityUtil.getUgiForKerberosUser(SecurityUtil.java:109) > > at > > org.apache.nifi.util.hive.HiveConfigurator.authenticate(HiveConfigurator.java:92) > > ... 17 common frames omitted > > Caused by: javax.security.auth.login.LoginException: Unable to login with > > ******************* due to: Message stream modified (41) > > at > > com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:808) > > at > > com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618) > > at sun.reflect.GeneratedMethodAccessor226.invoke(Unknown Source) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > > javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) > > at > > javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) > > at > > javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) > > at > > javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) > > at java.security.AccessController.doPrivileged(Native Method) > > at > > javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) > > at > > javax.security.auth.login.LoginContext.login(LoginContext.java:587) > > at > > org.apache.nifi.security.krb.AbstractKerberosUser.login(AbstractKerberosUser.java:81) > > at > > org.apache.nifi.hadoop.SecurityUtil.getUgiForKerberosUser(SecurityUtil.java:96) > > ... 18 common frames omitted > > > > Thanks again. > > > > Regards. > > > > Guille > > > > > > El jue, 27 ene 2022 a las 17:01, Joe Witt (<[email protected]>) escribió: > >> > >> Guille > >> > >> We are trying to be broadly compatible with every Java 8 and Java 11 > >> release we can but of course the older they get certain ones become > >> unusable and the newer they get sometimes new behavior are introduced. > >> We test a lot of combinations directly plus we hear a lot from threads > >> like this. But we simply can't know all/verify combinations. So > >> generally the answer is 'it should work' but of course sometimes > >> pieces break as the JVM changes. > >> > >> In this case you'll need to tell us more about your configuration for > >> us to really try/consider much. We'd need to hear about how you use > >> Kerb(do you use cross realm?) and we'd need to see the actual error's > >> you're seeing. Also why go to 282 now if there are much newer > >> versions available? I'm not sure about openjdk and its supported > >> status in Java 8. But you might want to also look at Azul or other > >> JDK providers. > >> > >> Thanks > >> > >> On Thu, Jan 27, 2022 at 8:40 AM Guillermo Muñoz Salgado > >> <[email protected]> wrote: > >> > > >> > Hi all, > >> > > >> > We are upgrading Java from OpenJDK1.8.222 to OpenJDK1.8.292, and > >> > everything seems to be ok except the Kerberos Controller Services. We > >> > think this issue [1] can be related. To mitigate it we launch NiFi with > >> > the next property in the bootstrap.conf file: > >> > java.arg.17=-Dsun.security.krb5.disableReferrals=true, but we get the > >> > same results. > >> > > >> > Are Kerberos Controller Services compatible with OpenJDK 1.8.282? > >> > Anyone else with similar problems out there? > >> > > >> > I paste our bootstrap.conf: > >> > > >> > > >> > java=java > >> > > >> > preserve.environment=false > >> > > >> > lib.dir=./lib > >> > > >> > conf.dir=./conf > >> > > >> > graceful.shutdown.seconds=20 > >> > > >> > java.arg.1=-Dorg.apache.jasper.compiler.disablejsr199=true > >> > > >> > java.arg.2=-Xms4G > >> > > >> > java.arg.3=-Xmx8G > >> > > >> > java.arg.4=-Djava.net.preferIPv4Stack=true > >> > > >> > java.arg.5=-Dsun.net.http.allowRestrictedHeaders=true > >> > > >> > java.arg.6=-Djava.protocol.handler.pkgs=sun.net.www.protocol > >> > > >> > java.arg.13=-XX:+UseG1GC > >> > > >> > java.arg.14=-Djava.awt.headless=true > >> > > >> > nifi.bootstrap.sensitive.key= > >> > > >> > java.arg.15=-Djava.security.egd=file:/dev/urandom > >> > > >> > java.arg.16=-Djavax.security.auth.useSubjectCredsOnly=true > >> > > >> > java.arg.17=-Dsun.security.krb5.disableReferrals=true > >> > > >> > java.arg.18=-Dzookeeper.admin.enableServer=true > >> > > >> > notification.services.file=./conf/bootstrap-notification-services.xml > >> > > >> > notification.max.attempts=5 > >> > > >> > java.arg.curator.supress.excessive.logs=-Dcurator-log-only-first-connection-issue-as-error-level=true > >> > > >> > > >> > Thanks in advance, > >> > -- > >> > Guille > >> > > >> > [1] https://bugs.openjdk.java.net/browse/JDK-8233512 > > > > > > > > -- > > Guille
