[jira] [Comment Edited] (NIFI-12918) Stateless NiFi NullPointerException on versioned PGs

2024-03-27 Thread Stephanie Ambrose (Jira)


[ 
https://issues.apache.org/jira/browse/NIFI-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17831362#comment-17831362
 ] 

Stephanie Ambrose edited comment on NIFI-12918 at 3/27/24 2:05 PM:
---

Okay, so after some back and forth here is my final conclusion to this bug:

I've submitted two PRs 
[https://github.com/apache/nifi/pull/8572/checks] <-- for support/1.x branch
[https://github.com/apache/nifi/pull/8536/checks] <-- for main branch

Here is what's happening. If you run a versioned stateless flow that has 
sub-versioned progress group, there is a NullPointerException thrown on the 
build method of StandardVersionControlInformation where there is a non-null 
requirement on "registryId". 

Code has changed in main versus support/1.x, but the bug exists on both. What 
I've found is the JerseyClient calls that are made to map the response from the 
registry-api back to the higher level VersionedProcessGroup -> 
VersionedFlowCoordinates both return JSON where the VersionedFlowCoordinates > 
registryId is always null. Now in the main branch, stateless uses some of these 
new "synchronizer" classes, so the best place to insert a solution is to add an 
else on the null check where the code runs the "determineRegistryId" method and 
set the value to "1" for versioned flows that do not have a registryId 
associated to them. Doing this passes all of the integration tests and code 
checks and fixes the bug. Since this class does not exist in the 1.x support 
branch, my best solution is to just comment out the null check (this breaks 
integration tests in main, but not in 1.x). 

I continued to look further at this idea of a "registryId" within the registry 
api code. The GET method on buckets/flow/version returns the same class that 
the stateless code uses to map into on the JerseyClient call. I then looked at 
the api code to POST new versioned flows, and it also expect the json as a 
parameter to match the same class. Well, since there is no "non-null" 
requirement on the registryId, versioned flows are stored in the database (or 
whichever storage adapter used) with this property as null. I did not check to 
see if the latest NiFi Registry UI is now setting this property, but making it 
non-null would break any older version of registry and not be backwards 
compatible. Therefore, the only solution at this point to fix stateless NiFi in 
its current state is to just set registryId to "1" if it is null. I believe 
this code is still probably prototype being worked and evolving, so this will 
be a temporary fix until those mandates are in place on a concept of 
"registryId". 

 

[~markap14] 


was (Author: JIRAUSER299049):
Okay, so after some back and forth here is my final conclusion to this bug:

I've submitted two PRs 
[https://github.com/apache/nifi/pull/8572/checks] <-- for support/1.x branch
[https://github.com/apache/nifi/pull/8536/checks] <-- for main branch

Here is what's happening. If you run a versioned stateless flow that has 
sub-versioned progress group, there is a NullPointerException thrown on the 
build method of StandardVersionControlInformation where there is a non-null 
requirement on "registryId". 

Code has changed in main versus support/1.x, but the bug exists on both. What 
I've found is the JerseyClient calls that are made to map the response from the 
registry-api back to the higher level VersionedProcessGroup -> 
VersionedFlowCoordinates both return JSON where the VersionedFlowCoordinates > 
registryId is always null. Now in the main branch, stateless uses some of these 
new "synchronizer" classes, so the best place to insert a solution is to add an 
else on the null check where the code runs the "determineRegistryId" method and 
set the value to "1" for versioned flows that do not have a registryId 
associated to them. Doing this passes all of the integration tests and code 
checks and fixes the bug. Since this class does not exist in the 1.x support 
branch, my best solution is to just comment out the null check (this breaks 
integration tests in main, but not in 1.x). 

I continued to look further at this idea of a "registryId" within the registry 
api code. The GET method on buckets/flow/version returns the same class that 
the stateless code uses to map into on the JerseyClient call. I then looked at 
the api code to POST new versioned flows, and it also expect the json as a 
parameter to match the same class. Well, since there is no "non-null" 
requirement on the registryId, versioned flows are stored in the database (or 
whichever storage adapter used) with this property as null. I did not check to 
see if the latest NiFi Registry UI is now setting this property, but making it 
non-null would break any older version of registry and not be backwards 
compatible. Therefore, the only solution at this point to fix stateless NiFi in 
its current state is to just set registryId to 

[jira] [Comment Edited] (NIFI-12918) Stateless NiFi NullPointerException on versioned PGs

2024-03-19 Thread Stephanie Ambrose (Jira)


[ 
https://issues.apache.org/jira/browse/NIFI-12918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828469#comment-17828469
 ] 

Stephanie Ambrose edited comment on NIFI-12918 at 3/19/24 5:54 PM:
---

[~dstiegli1] - Yes, I believe so. I'm submitting separate patch to support the 
main branch now. (same change)


was (Author: JIRAUSER299049):
Submitting separate patch to support the main branch.

> Stateless NiFi NullPointerException on versioned PGs
> 
>
> Key: NIFI-12918
> URL: https://issues.apache.org/jira/browse/NIFI-12918
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Stephanie Ambrose
>Assignee: Stephanie Ambrose
>Priority: Major
> Fix For: 1.26.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> If you create a process group and commit that to version control, and then 
> add a sub-process group under that one which is also under a separate version 
> control, it will throw a NullPointerException.
>  
> {code:java}
> 2024-03-15T12:25:38.565156967Z 2024-03-15 12:25:38,564 INFO [main] 
> o.a.nifi.groups.StandardProcessGroup 
> StandardProcessGroup[identifier=9688e64b-3f4a-3668-81b9-e25d9ea61454,name=Sub 
> Group] added to StandardProcessGroup[identifier=stateless-flow,name=Test Flow]
> 2024-03-15T12:25:38.570693223Z Exception in thread "main" 
> java.lang.NullPointerException: Registry ID must be specified
>   at java.base/java.util.Objects.requireNonNull(Unknown Source)
> 2024-03-15T12:25:38.570722016Zat 
> org.apache.nifi.registry.flow.StandardVersionControlInformation$Builder.build(StandardVersionControlInformation.java:134)
> 2024-03-15T12:25:38.570725111Zat 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:354)
> 2024-03-15T12:25:38.570729455Zat 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.addProcessGroup(StandardVersionedComponentSynchronizer.java:1185)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:528)
> 2024-03-15T12:25:38.573083029Zat 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:426)
> 2024-03-15T12:25:38.573086447Zat 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:265)
> 2024-03-15T12:25:38.573092747Zat 
> org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:551)
> 2024-03-15T12:25:38.573095677Zat 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:260)
> 2024-03-15T12:25:38.573098999Zat 
> org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3977)
>   at 
> org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3957)
>   at 
> org.apache.nifi.stateless.engine.StandardStatelessEngine.createFlow(StandardStatelessEngine.java:180)
> 2024-03-15T12:25:38.573108177Zat 
> org.apache.nifi.stateless.flow.StandardStatelessDataflowFactory.createDataflow(StandardStatelessDataflowFactory.java:243)
>   at 
> org.apache.nifi.stateless.bootstrap.StatelessBootstrap.createDataflow(StatelessBootstrap.java:73)
>  {code}
> When stepping through the debugger, it appears the "registryId" property is 
> null. The JerseyClient call is supposed to map the response from registry to 
> the proper class, but registryId is never set:
> {code:java}
> java.lang.Thread.State: RUNNABLE
> at 
> org.apache.nifi.flow.VersionedProcessGroup.setVersionedFlowCoordinates(VersionedProcessGroup.java:150)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Unknown 
> Source:-1)
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown 
> Source:-1)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown 
> Source:-1)
> at java.lang.reflect.Method.invoke(Unknown Source:-1)
> at 
> com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:141)
> at 
> com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:314)
> at 
> com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177)
> at 
> com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:359)
> at 
>