Hi Richard, I believe this is currently expected behavior, meaning that rolling deployments are not supported when changes to NARs are being made.
There are two main issues... The first is that if nodes are running different versions of the framework NAR, they could potentially be running incompatible versions of the REST API, which means if someone uses the UI while in this state, request replication could fail. This would be an issue even before 1.2.0, although I'm guessing you aren't using the UI during these deployments, or just not running into a case where the API was actually incompatible. The second is that we currently can't tell the difference between one of the nodes coming up with a new version of a custom NAR with the intent that you are about to upgrade the other nodes vs. one of the nodes coming up with a different version of a custom NAR because it was incorrectly deployed. As an example, imagine you have a 3 node cluster running v1 of your custom NAR, you stop everything intending to deploy v2 to all nodes, but you forget node3, and you start everything up. If we let all these nodes join the cluster, the user may have no idea that node3 is running the old code and possibly doing the wrong thing. All that being said, I do think we should figure out what can be done to support these types of rolling deployments, but I believe it will require a bit of design and work to make it happen. One idea that might be an option is to always deploy the new version of your custom NARs along side the existing version, this way the rolling restarts would work and your flow would remain running with the previous version, then upgrade the components in place through the UI. Admittedly this would currently be a bit tedious, but we could possibly look at ways of performing bulk upgrades. -Bryan On Fri, May 26, 2017 at 2:15 PM, Richard St. John <[email protected]> wrote: > Dear dev, > > I recently upgraded to NiFi version 1.2.0. In doing so, I lost the ability > to perform rolling upgrades on my 6-node cluster due to component > versioning, i.e. MissingBundleExceptions. My previous deployment steps are > listed below. My question for the devs is this: what is the best process > perform a rolling upgrade, with zero downtime, when custom nars, or NiFi > releases for that matter, need to be deployed. > > *Rolling deployment procedure with NiFi 1.1.x:* > 1. Package custom nars > 2. Run ansible script to copy nars to each node, and set a symlink, > custom-nars-current, to point to the new folder containing the newly built > custom nars. Note: the previous version of the nars are still on disk, but > the nifi.properties points to the symlink. > 3. Restart nifi service one node at a time. > -- > > ---------------------------- > Richard St. John, Ph.D. > Senior Software Engineer, Applied Mathematician > Asymmetrik, Ltd.
