[
https://issues.apache.org/jira/browse/AMBARI-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Tioseco updated AMBARI-10143:
-----------------------------------
Description:
The code that handles creating the data dir and reading data_dir_mount_file
explicitly converts the data dir paths to lower case.
This behavior is undesired because when the value of dfs.datanode.data.dir
contains one or more uppercase characters, it won't be able to match any item
in the data_dir_mount_file since Linux file paths are case sensitive.
The issue may be reproduced by creating a single node cluster and setting the
value of the "DataNode directories" in "CLUSTER INSTALL WIZARD > Customize
Services > HDFS > DataNode" to: /UPPER_CASE_DIR/data
Continue to the next step and the "DataNode start" task will fail with the
following exception (from /var/log/hadoop/hdfs/hadoop-hdfs-datanode-*.log):
{code}
datanode.DataNode (DataNode.java:checkStorageLocations(2284)) - Invalid
dfs.datanode.data.dir /UPPER_CASE_DIR/data :
java.io.FileNotFoundException: File file:/UPPER_CASE_DIR/data does not exist
{code}
The checkStorageLocations function won't be able to find the directory because
the directory name was created with all lower case characters (from
/var/lib/ambari-agent/data/output-9.txt):
{code}
Forcefully creating directory: /upper_case_dir/data
{code}
The data_dir_mount_file and hdfs-site.xml also shows the case mismatch of the
data dir path:
{code}
$ grep -rni UPPER_CASE_DIR /etc/hadoop/
/etc/hadoop/conf.empty/hdfs-site.xml:46:
<value>/UPPER_CASE_DIR/data</value>
/etc/hadoop/conf.empty/dfs_data_dir_mount.hist:7:/upper_case_dir/data,/
/etc/hadoop/conf/hdfs-site.xml:46: <value>/UPPER_CASE_DIR/data</value>
/etc/hadoop/conf/dfs_data_dir_mount.hist:7:/upper_case_dir/data,/
{code}
was:
The code that handles creating the data dir and reading data_dir_mount_file
explicitly converts the data dir paths to lower case.
This behavior is undesired because when the value of dfs.datanode.data.dir
contains one or more uppercase characters, it won't be able to match any item
in the data_dir_mount_file since Linux file paths are case sensitive.
The issue may be reproduced by creating a single node cluster and setting the
value of the "DataNode directories" in "CLUSTER INSTALL WIZARD > Customize
Services > HDFS > DataNode" to: /UPPER_CASE_DIR/data
Continue to the next step and the "DataNode start" task will fail with the
following exception (from /var/log/hadoop/hdfs/hadoop-hdfs-datanode-*.log):
```
datanode.DataNode (DataNode.java:checkStorageLocations(2284)) - Invalid
dfs.datanode.data.dir /UPPER_CASE_DIR/data :
java.io.FileNotFoundException: File file:/UPPER_CASE_DIR/data does not exist
```
The checkStorageLocations function won't be able to find the directory because
the directory name was created with all lower case characters (from
/var/lib/ambari-agent/data/output-9.txt):
```
Forcefully creating directory: /upper_case_dir/data
```
The data_dir_mount_file and hdfs-site.xml also shows the case mismatch of the
data dir path:
```
$ grep -rni UPPER_CASE_DIR /etc/hadoop/
/etc/hadoop/conf.empty/hdfs-site.xml:46:
<value>/UPPER_CASE_DIR/data</value>
/etc/hadoop/conf.empty/dfs_data_dir_mount.hist:7:/upper_case_dir/data,/
/etc/hadoop/conf/hdfs-site.xml:46: <value>/UPPER_CASE_DIR/data</value>
/etc/hadoop/conf/dfs_data_dir_mount.hist:7:/upper_case_dir/data,/
```
> DataNode fails to start because of lowercase conversion of data dir path in
> data_dir_mount_file.
> ------------------------------------------------------------------------------------------------
>
> Key: AMBARI-10143
> URL: https://issues.apache.org/jira/browse/AMBARI-10143
> Project: Ambari
> Issue Type: Bug
> Components: ambari-agent
> Affects Versions: 1.7.0
> Environment: Ubuntu 12.04 x64
> Reporter: Paulo Tioseco
> Attachments: AMBARI-10143.patch
>
>
> The code that handles creating the data dir and reading data_dir_mount_file
> explicitly converts the data dir paths to lower case.
> This behavior is undesired because when the value of dfs.datanode.data.dir
> contains one or more uppercase characters, it won't be able to match any item
> in the data_dir_mount_file since Linux file paths are case sensitive.
> The issue may be reproduced by creating a single node cluster and setting the
> value of the "DataNode directories" in "CLUSTER INSTALL WIZARD > Customize
> Services > HDFS > DataNode" to: /UPPER_CASE_DIR/data
> Continue to the next step and the "DataNode start" task will fail with the
> following exception (from /var/log/hadoop/hdfs/hadoop-hdfs-datanode-*.log):
> {code}
> datanode.DataNode (DataNode.java:checkStorageLocations(2284)) - Invalid
> dfs.datanode.data.dir /UPPER_CASE_DIR/data :
> java.io.FileNotFoundException: File file:/UPPER_CASE_DIR/data does not exist
> {code}
> The checkStorageLocations function won't be able to find the directory
> because the directory name was created with all lower case characters (from
> /var/lib/ambari-agent/data/output-9.txt):
> {code}
> Forcefully creating directory: /upper_case_dir/data
> {code}
> The data_dir_mount_file and hdfs-site.xml also shows the case mismatch of the
> data dir path:
> {code}
> $ grep -rni UPPER_CASE_DIR /etc/hadoop/
> /etc/hadoop/conf.empty/hdfs-site.xml:46:
> <value>/UPPER_CASE_DIR/data</value>
> /etc/hadoop/conf.empty/dfs_data_dir_mount.hist:7:/upper_case_dir/data,/
> /etc/hadoop/conf/hdfs-site.xml:46: <value>/UPPER_CASE_DIR/data</value>
> /etc/hadoop/conf/dfs_data_dir_mount.hist:7:/upper_case_dir/data,/
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
