[
https://issues.apache.org/jira/browse/NIFI-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17754124#comment-17754124
]
Bryan Bende commented on NIFI-11927:
------------------------------------
It looks like registry 1.20.0 uses sshd 2.9.2 and somewhere after that it was
upgrade to 2.10.0, but the jgit version has remained the same from 1.20.0 to
1.23.0 and that version is 5.13.1.202206130422, so I'm guessing this jgit
version is not compatible with sshd 2.10.0.
> NiFi Registry fails to push flow to GIT using SSH
> -------------------------------------------------
>
> Key: NIFI-11927
> URL: https://issues.apache.org/jira/browse/NIFI-11927
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Registry
> Affects Versions: 1.22.0, 1.23.0
> Environment: Java 17 & 11, CentOS 9, kernel 5.14.0-44.el9.x86_64
> Reporter: Daniel Salwerowicz
> Priority: Major
> Attachments: image-2023-08-10-08-53-15-944.png
>
>
> I have been successfully running NiFi Registry version 1.20.0 (and others
> before it) for about a year now. Now I needed to switch server that Registry
> runs on, since the old one was outdated, and decided to set up NiFi Registry
> version 1.22.0.
> The setup worked great and NiFi seems to have been running properly. However
> I recently noticed that it wasn't pushing anything to GIT. The changes are
> made, new versions are committed, they show up both in Registry GUI as well
> as the NiFi instance, but when I try to fetch some version it will not allow
> me giving me the following error:
> !image-2023-08-10-08-53-15-944.png!
> I started investigating the issue and noticed that the GIT repository that
> NiFi Registry was using was several versions ahead of origin, no version got
> pushed to the GIT repo.
> I investigated the logs and noticed following messages show up when I make
> new version in NiFi and commit it to registry:
> {code:java}
> 2023-08-10 07:52:03,317 WARN [NiFi Registry Web Server-18]
> javax.persistence.spi javax.persistence.spi::No valid providers found.
> 2023-08-10 07:52:05,120 INFO [GitFlowMetaData Push thread]
> o.a.s.c.u.s.e.EdDSASecurityProviderRegistrar getOrCreateProvider(EdDSA)
> created instance of net.i2p.crypto.eddsa.EdDSASecurityProvider
> 2023-08-10 07:52:05,326 INFO [GitFlowMetaData Push thread]
> o.a.s.c.i.DefaultIoServiceFactoryFactory No detected/configured
> IoServiceFactoryFactory; using Nio2ServiceFactoryFactory
> 2023-08-10 07:52:05,351 ERROR [GitFlowMetaData Push thread]
> o.a.n.r.p.flow.git.GitFlowMetaData Failed to push commits to origin due to
> org.eclipse.jgit.api.errors.TransportException:
> git@<OMITTED_FOR_PRIVACY>nifi/nifi-flows.git: remote hung up unexpectedly
> org.eclipse.jgit.api.errors.TransportException:
> git@<OMITTED_FOR_PRIVACY>nifi/nifi-flows.git: remote hung up unexpectedly
> at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:147)
> at
> org.apache.nifi.registry.provider.flow.git.GitFlowMetaData.lambda$startPushThread$1(GitFlowMetaData.java:299)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
> at
> java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:833)
> Caused by: org.eclipse.jgit.errors.TransportException:
> [email protected]:digital-bevaring/nifi/nifi-flows.git: remote hung up
> unexpectedly
> at
> org.eclipse.jgit.transport.TransportGitSsh$SshPushConnection.<init>(TransportGitSsh.java:382)
> at
> org.eclipse.jgit.transport.TransportGitSsh.openPush(TransportGitSsh.java:159)
> at
> org.eclipse.jgit.transport.PushProcess.execute(PushProcess.java:127)
> at org.eclipse.jgit.transport.Transport.push(Transport.java:1384)
> at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:137)
> ... 7 common frames omitted
> Caused by: java.lang.NoSuchMethodError: 'java.lang.Object
> org.apache.sshd.client.future.ConnectFuture.verify()'
> at
> org.eclipse.jgit.transport.sshd.SshdSession.connect(SshdSession.java:189)
> at
> org.eclipse.jgit.transport.sshd.SshdSession.connect(SshdSession.java:142)
> at
> org.eclipse.jgit.transport.sshd.SshdSession.connect(SshdSession.java:99)
> at
> org.eclipse.jgit.transport.sshd.SshdSessionFactory.getSession(SshdSessionFactory.java:235)
> at
> org.eclipse.jgit.transport.sshd.SshdSessionFactory.getSession(SshdSessionFactory.java:1)
> at
> org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:107)
> at
> org.eclipse.jgit.transport.TransportGitSsh$SshPushConnection.<init>(TransportGitSsh.java:358)
> ... 11 common frames omitted {code}
> I think the line {{Caused by: java.lang.NoSuchMethodError: 'java.lang.Object
> org.apache.sshd.client.future.ConnectFuture.verify()'}} is what's causing the
> issue with pushing to GIT. This does not show up on the old Registry instance
> when I push to that.
>
> I checked the setup and compared it to the previous version running on old
> server. They have exactly the same configuration, save for new one running on
> Java 17 not 11 (although I tried running it on 11 too).
>
> The SSH keys are valid and recognized by our Gitlab instance, I can push
> manually as the nifi user, there are literally no differences between the
> configuration on one server compared to another. I even checked the
> configuration inside .git folder in the flow_storage directory, it was the
> same as on the other server. The ~/.ssh folder for nifi user has similar data
> between two servers, both have keys and known hosts set up, with right
> permissions, etc.
> I decided to try running version 1.20.0 of registry on new server, and it
> seems to have worked immediately, without any issue, it manages to push to
> GIT on its own, with no changes to config. I tested version 1.23 as well and
> it had the same issue, I haven't tested 1.21.0 though.
>
> For more information, here is the structure I have:
> {code:java}
> ├── nifi-registry-1.20.0
> │ ├── bin
> │ ├── conf
> │ ├── docs
> │ ├── ext
> │ ├── lib
> │ ├── logs
> │ ├── run
> │ └── work
> ├── nifi-registry-1.23.0
> │ ├── bin
> │ ├── conf
> │ ├── docs
> │ ├── ext
> │ ├── lib
> │ ├── logs
> │ ├── run
> │ └── work
> └── nifi-registry-files
> ├── authorization-files
> │ ├── authorizations.xml
> │ ├── authorizers.xml
> │ ├── login-identity-providers.xml
> │ └── users.xml
> ├── certificate-files
> │ ├── keystore.jks
> │ └── truststore.jks
> ├── configuration-files
> │ ├── providers.xml
> │ └── registry-aliases.xml
> ├── database-drivers
> │ └── mariadb-java-client-2.7.4.jar
> ├── extension-bundles
> └── flow-storage
> └── Buckets... {code}
>
> Contents of the providers.xml:
> {code:java}
> <providers>
> <flowPersistenceProvider>
>
> <class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class>
> <property name="Flow Storage
> Directory">/disk1/nifi-registry/nifi-registry-files/flow-storage</property>
> <property name="Remote To Push">origin</property>
> <property name="Remote Access User"></property>
> <property name="Remote Access Password"></property>
> <property name="Remote Clone Repository"></property>
> </flowPersistenceProvider>
> <extensionBundlePersistenceProvider>
>
> <class>org.apache.nifi.registry.provider.extension.FileSystemBundlePersistenceProvider</class>
> <property name="Extension Bundle Storage
> Directory">/disk1/nifi-registry/nifi-registry-files/extension-bundles</property>
> </extensionBundlePersistenceProvider>
> </providers>{code}
>
> Contents of the .git/config file in the flow_storage directory:
> {code:java}
> [core]
> repositoryformatversion = 0
> filemode = true
> bare = false
> logallrefupdates = true
> [http]
> sslVerify = false
> proxy = http://<PROXY_URL>:3128
> [https]
> sslVerify = false
> proxy = http://<PROXY_URL>:3128
> [user]
> name = nifi_user
> email = OMITTED_FOR_PRIVACY
> [remote "origin"]
> url = git@<OMITTED_FOR_PRIVACY>nifi/nifi-flows.git
> fetch = +refs/heads/*:refs/remotes/origin/*
> [branch "master"]
> remote = origin
> merge = refs/heads/master {code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
