I believe I missed a crucial file — if you are using the default file-based authorizer provided by NiFi, you will also want to copy conf/authorized-users.xml which defines the various users and their access control policies. Sorry about that, the instance I referenced when typing wasn’t using it so I forgot.
Andy LoPresto [email protected] [email protected] PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jun 29, 2017, at 5:17 PM, Andy LoPresto <[email protected]> wrote: > > Jeremy, > > Sorry to hear you are having difficulties upgrading. I’ll try to answer your > questions as best I can, but welcome others joining the thread. > > It sounds like this is less of an in-place upgrade and more of a cluster > migration since you are installing the software on new nodes and moving the > flow there. If you do not also move the various repositories, you effectively > have a new fresh install and have just copied the flow definition. > > The Apache NiFi wiki does provide Migration Guidance [1] and this document is > updated with best practices/standard operating procedure with each release. > It is prepared by the developers who have tested the upgrade process, and > hopefully is helpful to end users doing the same. Perhaps even more relevant > is the Upgrade Guide [2], which has step-by-step instructions for configuring > an instance to be easily upgradable, and how to perform the upgrades in-place > quickly and repeatably. > > To “upgrade” to a new node and maintain the existing functionality and > configurations, you would want to copy additional files: > > * conf/authorizers.xml — contains the user/group providers and access policy > providers (separated during 1.3.0) which control permissions in the > application > * conf/bootstrap.conf — contains the master key for encryption and other NiFi > runtime arguments > * conf/logback.xml — defines log levels > * conf/login-identity-providers.xml — defines LDAP or Kerberos connection > details > * conf/nifi.properties — global property definition for the application > * conf/state-management.xml — defines state management configuration > (especially important for a cluster) > * conf/zookeeper.properties — defines ZK configurations and addresses > (especially important for a cluster) > > [Optional] > > * flowfile_repository/ — copy this to allow flowfile replay and history > * content_repository/ — copy this to allow flowfile replay and history > * provenance_repository/ — copy this to maintain the history of the > provenance records > > I’m not quite sure what you mean by “keeping the state of my consumers” — is > this specifically a ConsumeKafka processor or just source processors in > general? If you upgrade in-place or copy the various configs and repositories > mentioned above, when the app starts again, the component state should be as > you left it (barring code changes to the component in the new release). > > Hopefully those instructions and the documents linked below will help you. We > definitely want to make upgrading a seamless experience because we are > constantly working to bring new features and optimizations into the app, and > supporting legacy versions is an added cost. We are not looking for version > fragmentation here. Any suggestions you have for improving the upgrade > process and our communication around those instructions is welcome. Thanks. > > [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance > <https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance> > [2] https://cwiki.apache.org/confluence/display/NIFI/Upgrading+NiFi > <https://cwiki.apache.org/confluence/display/NIFI/Upgrading+NiFi> > > > Andy LoPresto > [email protected] <mailto:[email protected]> > [email protected] <mailto:[email protected]> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On Jun 29, 2017, at 4:15 PM, Jeremy Farbota <[email protected] >> <mailto:[email protected]>> wrote: >> >> We are attempting to upgrade and we're having issues: >> >> We created new nodes with the new version (1.3.0). We stopped all flowed and >> ensured nothing is in queue. We copied the flow.xml.gz file to the new node. >> >> Are following the right protocol? Should we be copying the users.xml file as >> well? >> >> Is there a way to migrate to the new version while keeping the state of my >> consumers? >> >> It seems like I have to build a cluster and recreate the user permissions >> and everything from scratch. Is there a workaround? >> >> Is there a wiki or any notes about how to upgrade a cluster? >> >> Last time I posted this there was a reply with a link to a thread that does >> not provide any additional info about how to bring the cluster up with the >> new version. >> >> >> Jeremy Farbota >> Software Engineer, Data >> Payoff, Inc. >> >> [email protected] <mailto:[email protected]> >> (217) 898-8110 <tel:+2178988110> >> On Thu, Jun 15, 2017 at 12:23 PM, Jeremy Farbota <[email protected] >> <mailto:[email protected]>> wrote: >> We're preparing to upgrade our cluster from 1.0.0 to 1.3.0. >> >> We're using external zookeeper. >> >> We're wondering if we can simply expand the cluster with 1.3.0 machines and >> then turn off the other ones to keep processes running. Are there issues >> with attempting that? Is there an upgrade guide someone for clusters on Wiki? >> >> We have kafka consumers and other maintenance processes that are running in >> production so we'd like to make the change without messing with the state of >> those consumers if possible. >> >> Kindly, >> >> >> >> Jeremy Farbota >> Software Engineer, Data >> Payoff, Inc. >> >> [email protected] <mailto:[email protected]> >> (217) 898-8110 <tel:+2178988110> >
signature.asc
Description: Message signed with OpenPGP using GPGMail
