----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/40377/ -----------------------------------------------------------
Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Nate Cole, and Swapan Shridhar. Repository: ambari Description ------- In HDP 2.3.4.0, the the structure of stored acls for Kafka in zookeeper has changed. This means to support upgrade-to or downgrade-from HDP 2.3.4.0 we will have to move these stored acls to new format or vice versa. Ambari should make the following changes for upgrade/downgrade process of Kafka. PreReq: Install bits side-by-side Steps: 1) Stop all Kafka brokers 2) Call bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=localhost:2181 --upgradeAcls or bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=localhost:2181 --downgradeAcls 3) Start all Kafka brokers. Example Cases: 1) If upgrading from HDP 2.3.0.0 -> 2.3.2.0, we will still stop all Kafka brokers since cannot distinguish the version in the single upgrade pack. Customers may modify this behavior by editing upgrade-2.3.xml. Because the 2.3.4.0 boundary is not crossed, we should not call the migration script. 2) If upgrading from HDP 2.3.4.0->2.3.5.0, then we will stop all Kafka brokers and not call the migration script since not crossing the boundary. 3) If upgrading from HDP 2.3.2.0->2.3.5.0, then we will stop all Kafka brokers and call the migration script since crossing the boundary. On downgrade, we should call the migration script again. Diffs ----- ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py 8efd564 ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py dc3a61b ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/upgrade.py 22036f7 ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml fe87ed2 ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 90f50a4 Diff: https://reviews.apache.org/r/40377/diff/ Testing ------- Tested in HDP 2.2 with Kafka, then EU from 2.2 -> 2.3.4.0 called migration script, then added NameNode HA, and RU from 2.3.4.0-1 to 2.3.4.0-2 did not call script HDP 2.2 with kafka, then added NameNode HA, and RU from HDP 2.2->2.3.4.0-1, which called migration script, followed by RU from 2.3.4.0-1 to 2.3.4.0-2 which did not call script Python unit tests passed, ---------------------------------------------------------------------- Total run:829 Total errors:0 Total failures:0 OK Thanks, Alejandro Fernandez
