-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38788/#review100819
-----------------------------------------------------------
Can you update dfs_datanode_helper.py as well?
ambari-common/src/main/python/resource_management/libraries/functions/dfs_datanode_helper.py:
This function relies on the dfs.datanode.data.dir.mount.file parameter to
parse a file that contains
ambari-common/src/main/python/resource_management/libraries/functions/dfs_datanode_helper.py:
Logger.warning("DataNode is allowed to create any data directory since
dfs.datanode.data.dir.mount.file property is null.")
ambari-common/src/main/python/resource_management/libraries/functions/dfs_datanode_helper.py:
Logger.warning("DataNode is allowed to create any data directory since
dfs.datanode.data.dir.mount.file property has file %s and it does not exist." %
params.data_dir_mount_file)
- Jayush Luniya
On Sept. 28, 2015, 5:29 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38788/
> -----------------------------------------------------------
>
> (Updated Sept. 28, 2015, 5:29 a.m.)
>
>
> Review request for Ambari, Dmytro Sen, Jayush Luniya, Mahadev Konar, and
> Sumit Mohanty.
>
>
> Bugs: AMBARI-13251
> https://issues.apache.org/jira/browse/AMBARI-13251
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During the Clients group of RU, restarting hdfs_client changes changes the
> symlink /usr/hdp/current/hadoop-client/ from the old version to the new
> version.
>
> Because the Data Dir Mount History file is stored at
> /etc/hadoop/conf/dfs_data_dir_mount.hist and
> /etc/hadoop/conf symlinks to /usr/hdp/current/hadoop-client/conf
> it means that the file must be copied to the new location.
>
> Multiple clients (HDFS, HBase, Hive, YARN, MapReduce, Pig, Slider, Tez)
> actually end up calling
> {code}
> hdp_select.select("hadoop-client", params.version)
> {code}
> that changes the symlink, so the first one to be ran during RU must copy the
> file. If DataNode is present, it means that HDFS Client is guaranteed to be
> there, and from all of those clients it is the first one in the Upgrade Pack.
>
> During a Rolling Downgrade, the file should already exist in the previous
> version, and the file is only written to when DataNode is restarted, so we
> are ok.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/Controller.py 8746172
> ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py
> 70539ac
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog212.java
> 37a87ab
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml
> 4cb2274
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/alerts/alert_datanode_unmounted_data_dir.py
> 2912406
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py
> 34ec8cd
>
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
> 3160af2
>
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/configuration/hadoop-env.xml
> c6dd202
>
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/package/scripts/params.py
> fb0a4db
>
> ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/HDFS/configuration/hadoop-env.xml
> b5451d8
>
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_alert_datanode_unmounted_data_dir.py
> 4406231
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py d5a42f0
> ambari-server/src/test/python/stacks/2.0.6/configs/altfs_plus_hdfs.json
> f6c9bb4
> ambari-server/src/test/python/stacks/2.0.6/configs/client-upgrade.json
> c13e5c9
> ambari-server/src/test/python/stacks/2.0.6/configs/default.json a0da7f0
> ambari-server/src/test/python/stacks/2.0.6/configs/default_hive_nn_ha.json
> b644411
>
> ambari-server/src/test/python/stacks/2.0.6/configs/default_hive_nn_ha_2.json
> 770e085
>
> ambari-server/src/test/python/stacks/2.0.6/configs/default_hive_non_hdfs.json
> 988f38e
> ambari-server/src/test/python/stacks/2.0.6/configs/default_no_install.json
> 4cf647b
> ambari-server/src/test/python/stacks/2.0.6/configs/default_oozie_mysql.json
> 286a728
>
> ambari-server/src/test/python/stacks/2.0.6/configs/default_update_exclude_file_only.json
> b2fd6e8
> ambari-server/src/test/python/stacks/2.0.6/configs/flume_22.json 4f30caa
> ambari-server/src/test/python/stacks/2.0.6/configs/hbase-2.2.json e7a516f
> ambari-server/src/test/python/stacks/2.0.6/configs/hbase-check-2.2.json
> 83120eb
> ambari-server/src/test/python/stacks/2.0.6/configs/hbase-rs-2.2.json
> 349a1cf
> ambari-server/src/test/python/stacks/2.0.6/configs/hbase_no_phx.json
> b71c4c8
> ambari-server/src/test/python/stacks/2.0.6/configs/hbase_with_phx.json
> a7187f9
> ambari-server/src/test/python/stacks/2.0.6/configs/nn_ru_lzo.json a026259
> ambari-server/src/test/python/stacks/2.0.6/configs/oozie_existing_sqla.json
> 01856df
>
> ambari-server/src/test/python/stacks/2.0.6/configs/ranger-namenode-start.json
> 84c798b
> ambari-server/src/test/python/stacks/2.0.6/configs/secured.json c9e20e2
> ambari-server/src/test/python/stacks/2.1/configs/client-upgrade.json
> ca0b1a6
> ambari-server/src/test/python/stacks/2.2/configs/default.json dcd38d6
>
> ambari-server/src/test/python/stacks/2.2/configs/default_custom_path_config.json
> 2c9c918
> ambari-server/src/test/python/stacks/2.2/configs/falcon-upgrade.json
> a4f9dc9
> ambari-server/src/test/python/stacks/2.2/configs/hive-upgrade.json 95456f9
>
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade-hdfs-secure.json
> e06882a
> ambari-server/src/test/python/stacks/2.2/configs/journalnode-upgrade.json
> 1f23f11
> ambari-server/src/test/python/stacks/2.2/configs/knox_upgrade.json c77b3ec
> ambari-server/src/test/python/stacks/2.2/configs/oozie-downgrade.json
> e9018d8
> ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json fc4df58
>
> ambari-server/src/test/python/stacks/2.2/configs/pig-service-check-secure.json
> ad7a75a
> ambari-server/src/test/python/stacks/2.2/configs/ranger-admin-default.json
> f012a17
> ambari-server/src/test/python/stacks/2.2/configs/ranger-admin-upgrade.json
> f47a247
>
> ambari-server/src/test/python/stacks/2.2/configs/ranger-usersync-upgrade.json
> fa4d614
>
> ambari-server/src/test/python/stacks/2.2/configs/spark-job-history-server.json
> 185d4d5
> ambari-server/src/test/python/stacks/2.3/configs/hbase_default.json f5a4487
> ambari-server/src/test/python/stacks/2.3/configs/hbase_secure.json d31c621
> ambari-server/src/test/python/stacks/2.3/configs/spark_default.json 730a81b
>
> ambari-server/src/test/resources/custom_actions/ru_execute_tasks_namenode_prepare.json
> d42d01c
> ambari-web/app/assets/data/configurations/config_versions.json 603d33e
> ambari-web/app/assets/data/configurations/service_versions.json c5eb46c
> ambari-web/app/assets/data/stacks/HDP-2.2/configurations.json 0399656
> ambari-web/app/data/BIGTOP/site_properties.js e71cab4
> ambari-web/app/data/HDP2/site_properties.js 03ffd31
>
> Diff: https://reviews.apache.org/r/38788/diff/
>
>
> Testing
> -------
>
> Tested the alert during RU, and the mount history file was copied to the new
> location, so no warnings were generated.
>
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:805
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
