> On June 21, 2017, 6:04 a.m., Sebastian Toader wrote: > > Run unit ambari-server unit tests and post results here. > > > > Perform manual testing of upgrade scenarios eg. install a cluster that has > > Kafka an older version (e.g. 2.4 or 2.5.1) of Ambari that doesn't have the > > feature for setting rack info for Kafka. Verify that Kakfa works as > > expected. Than upgrade Ambari on the existing cluster to higher version > > that has the feature for setting rack info for Kafka and verify that Kafka > > still works as expected. > > Ambud Sharma wrote: > Unit tests failed. If there's a specific flag I should be running them > with, please let me know. > > Manually tested Ambari version with and without the stack. > > > Tests run: 4277, Failures: 0, Errors: 0, Skipped: 32 > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main ........................................ SUCCESS [ > 11.418 s] > [INFO] Apache Ambari Project POM .......................... SUCCESS [ > 0.062 s] > [INFO] Ambari Web ......................................... SUCCESS > [03:50 min] > [INFO] Ambari Views ....................................... SUCCESS [ > 3.410 s] > [INFO] Ambari Admin View .................................. SUCCESS [ > 17.426 s] > [INFO] utility ............................................ SUCCESS [ > 3.040 s] > [INFO] ambari-metrics ..................................... SUCCESS [ > 2.070 s] > [INFO] Ambari Metrics Common .............................. SUCCESS [ > 7.596 s] > [INFO] Ambari Metrics Hadoop Sink ......................... SUCCESS [ > 8.955 s] > [INFO] Ambari Metrics Flume Sink .......................... SUCCESS [ > 6.631 s] > [INFO] Ambari Metrics Kafka Sink .......................... SUCCESS [ > 5.576 s] > [INFO] Ambari Metrics Storm Sink .......................... SUCCESS [ > 2.765 s] > [INFO] Ambari Metrics Storm Sink (Legacy) ................. SUCCESS [ > 2.231 s] > [INFO] Ambari Metrics Collector ........................... SUCCESS > [05:52 min] > [INFO] Ambari Metrics Monitor ............................. SUCCESS [ > 3.240 s] > [INFO] Ambari Metrics Grafana ............................. SUCCESS [ > 33.955 s] > [INFO] Ambari Metrics Host Aggregator ..................... SUCCESS [ > 7.216 s] > [INFO] Ambari Metrics Assembly ............................ SUCCESS [ > 18.381 s] > [INFO] Service Advisor .................................... SUCCESS [ > 1.301 s] > [INFO] Ambari Server ...................................... FAILURE > [32:36 min] > [INFO] Ambari Functional Tests ............................ SKIPPED > [INFO] Ambari Agent ....................................... SKIPPED > [INFO] Ambari Client ...................................... SKIPPED > [INFO] Ambari Python Client ............................... SKIPPED > [INFO] Ambari Groovy Client ............................... SKIPPED > [INFO] Ambari Shell ....................................... SKIPPED > [INFO] Ambari Python Shell ................................ SKIPPED > [INFO] Ambari Groovy Shell ................................ SKIPPED > [INFO] ambari-logsearch ................................... SKIPPED > [INFO] Ambari Logsearch Appender .......................... SKIPPED > [INFO] Ambari Logsearch Config Api ........................ SKIPPED > [INFO] Ambari Logsearch Config Zookeeper .................. SKIPPED > [INFO] Ambari Logsearch Web ............................... SKIPPED > [INFO] Ambari Logsearch Server ............................ SKIPPED > [INFO] Ambari Logsearch Log Feeder ........................ SKIPPED > [INFO] Ambari Logsearch Assembly .......................... SKIPPED > [INFO] Ambari Logsearch Integration Test .................. SKIPPED > [INFO] ambari-infra ....................................... SKIPPED > [INFO] Ambari Infra Solr Client ........................... SKIPPED > [INFO] Ambari Infra Solr Plugin ........................... SKIPPED > [INFO] Ambari Infra Manager ............................... SKIPPED > [INFO] Ambari Infra Solr Assembly ......................... SKIPPED > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 44:36 min > [INFO] Finished at: 2017-06-21T20:01:27Z > [INFO] Final Memory: 111M/1011M > [INFO] > ------------------------------------------------------------------------ > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-surefire-plugin:2.19:test (default-test) on > project ambari-server: ExecutionException The forked VM terminated without > properly saying goodbye. VM crash or System.exit called? > [ERROR] Command was /bin/sh -c cd /home/centos/ambari/ambari-server && > /usr/java/jdk1.8.0_112/jre/bin/java -Xmx1024m -XX:MaxPermSize=512m -Xms512m > -jar > /home/centos/ambari/ambari-server/target/surefire/surefirebooter3486762047752418417.jar > > /home/centos/ambari/ambari-server/target/surefire/surefire7489163180494317107tmp > > /home/centos/ambari/ambari-server/target/surefire/surefire_4482532294765351508tmp > > Sebastian Toader wrote: > Run unit tests only for ambari server: mvn clean test -am -pl > ambari-server
Failed tests: BufferedThreadPoolExecutorCompletionServiceTest.testMaxPoolSizeThreadsLaunched:143 expected:<10> but was:<9> Tests run: 4954, Failures: 1, Errors: 0, Skipped: 35 [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Ambari Main ........................................ SUCCESS [ 11.419 s] [INFO] Apache Ambari Project POM .......................... SUCCESS [ 0.089 s] [INFO] Ambari Views ....................................... SUCCESS [ 4.067 s] [INFO] utility ............................................ SUCCESS [ 3.593 s] [INFO] ambari-metrics ..................................... SUCCESS [ 1.401 s] [INFO] Ambari Metrics Common .............................. SUCCESS [ 8.732 s] [INFO] Service Advisor .................................... SUCCESS [ 0.728 s] [INFO] Ambari Server ...................................... FAILURE [48:12 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 48:44 min [INFO] Finished at: 2017-06-22T16:20:36Z [INFO] Final Memory: 68M/579M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19:test (default-test) on project ambari-server: There are test failures. [ERROR] [ERROR] Please refer to /home/centos/ambari/ambari-server/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :ambari-server > On June 21, 2017, 6:04 a.m., Sebastian Toader wrote: > > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py > > Line 104 (original), 104 (patched) > > <https://reviews.apache.org/r/60249/diff/1/?file=1754688#file1754688line104> > > > > According to https://issues.apache.org/jira/browse/KAFKA-1215 rack > > awarness only supported starting from version 0.10.0.0. Shouldn't this > > change be implemented in common-services/KAFKA/0.10.0 instead of > > common-services/KAFKA/0.8.1 ? > > > > If setting ```broker.rack``` can cause issues for older Kafka version > > that doesn't support rack awarness than the setting of ```broker.rack``` > > has to be added to the scripts of the appropriate Kafka version on Ambari > > stack definition or via stack_features. > > Ambud Sharma wrote: > Added a new patch that covers 0.10.0.3 common-services version of the > stack. The broker.rack property if set on older version of Kafka will simply > get ignored (verified with 0.8.2.2) > > Sebastian Toader wrote: > Please address my other comment with regards to topology script as well. > Users wanting to use their own custom logic for rack information will upload > their own topology script. Currently they have to deploy themselves to all > host their custom script but for Ambari 3.0.0 we're adding a feature that > will deploy user provided topology script to the hosts for them. I have commented on that below, as of Ambari 3.0.0 if that script will always be available in /etc/hadoop/ regardless of HDFS or YARN being installed then we can surely plan on how to include it. However, if that script doesn't get pushed out for Kafka install then it's difficult to enable that. - Ambud ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60249/#review178450 ----------------------------------------------------------- On June 21, 2017, 8:33 p.m., Ambud Sharma wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60249/ > ----------------------------------------------------------- > > (Updated June 21, 2017, 8:33 p.m.) > > > Review request for Ambari and Alejandro Fernandez. > > > Repository: ambari > > > Description > ------- > > Ambari rack awareness for Kafka. > https://issues.apache.org/jira/browse/AMBARI-21234 > > As an operations person it would be nice to manage Kafka rack awareness via > Ambari. Ambari allows node rack information to be configured and this > information can then be pulled in the Kafka stack and populated in the > server.properties file for Kafka. > Design: > This stack change uses the /clusterHostInfo/all_hosts and > /clusterHostInfo/all_racks paths and materializes them to a variable. Then it > uses linear search to find this node in the list of all hosts and it's > corresponding rack id. This information is then stored in a variable called > rack and which is materialized during the configure method of the broker > scripts. > This stack change relies on the node rack information stored in Ambari > therefore will enable both Ambari UI and Blueprints to be used for setting up > Kafka broker rack information. > > > Diffs > ----- > > > ambari-server/src/main/resources/common-services/KAFKA/0.10.0.3.0/package/scripts/kafka.py > 680dd32802 > > ambari-server/src/main/resources/common-services/KAFKA/0.10.0.3.0/package/scripts/params.py > 5b0be54982 > > > Diff: https://reviews.apache.org/r/60249/diff/2/ > > > Testing > ------- > > Manually deployed Kafka cluster and verified the broker.rack property is > correctly populated. > > > Thanks, > > Ambud Sharma > >
