[
https://issues.apache.org/jira/browse/HDFS-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ECFuzz updated HDFS-17069:
--------------------------
Description:
My hadoop version is 3.3.6, and I use the Pseudo-Distributed Operation.
`core-site.xml` like below.
{quote}```shell
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/Mutil_Component/tmp</value>
</property>
</configuration>
```
{quote}
`hdfs-site.xml` like below.
```shell
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.blocksize</name>
<value>128k</value>
</property>
</configuration>
```
And then format the namenode, and start the hdfs.
```shell
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
bin/hdfs namenode -format
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx(many info)
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
sbin/start-dfs.sh
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [hadoop-Standard-PC-i440FX-PIIX-1996]
```
Finally, use dfs to put a file. Then I get the message which means 128k is less
than 1M.
```shell
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
bin/hdfs dfs -mkdir -p /user/hadoop
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
bin/hdfs dfs -mkdir input
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
bin/hdfs dfs -put etc/hadoop/hdfs-site.xml input
put: Specified block size is less than configured minimum value
(dfs.namenode.fs-limits.min-block-size): 131072 < 1048576
```
But I find that in the document, dfs.blocksize can be set like 128k and other
values in `hdfs-default.xml` .
```shell
The default block size for new files, in bytes. You can use the following
suffix (case insensitive): k(kilo), m(mega), g(giga), t(tera), p(peta), e(exa)
to specify the size (such as 128k, 512m, 1g, etc.), Or provide complete size in
bytes (such as 134217728 for 128 MB).
```
So, should there be some issues with the documents here？Or should notice user
to set this configuration to be lager than 1M?
was:
My hadoop version is 3.3.6, and I use the Pseudo-Distributed Operation.
`core-site.xml` like below.
```shell
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/Mutil_Component/tmp</value>
</property>
</configuration>
```
`hdfs-site.xml` like below.
```shell
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.blocksize</name>
<value>128k</value>
</property>
</configuration>
```
And then format the namenode, and start the hdfs.
```shell
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
bin/hdfs namenode -format
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx(many info)
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
sbin/start-dfs.sh
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [hadoop-Standard-PC-i440FX-PIIX-1996]
```
Finally, use dfs to put a file. Then I get the message which means 128k is less
than 1M.
```shell
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
bin/hdfs dfs -mkdir -p /user/hadoop
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
bin/hdfs dfs -mkdir input
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
bin/hdfs dfs -put etc/hadoop/hdfs-site.xml input
put: Specified block size is less than configured minimum value
(dfs.namenode.fs-limits.min-block-size): 131072 < 1048576
```
But I find that in the document, dfs.blocksize can be set like 128k and other
values in `hdfs-default.xml` .
```shell
The default block size for new files, in bytes. You can use the following
suffix (case insensitive): k(kilo), m(mega), g(giga), t(tera), p(peta), e(exa)
to specify the size (such as 128k, 512m, 1g, etc.), Or provide complete size in
bytes (such as 134217728 for 128 MB).
```
So, should there be some issues with the documents here？Or should notice user
to set this configuration to be lager than 1M?
> The documentation and implementation of "dfs.blocksize" are inconsistent.
> -------------------------------------------------------------------------
>
> Key: HDFS-17069
> URL: https://issues.apache.org/jira/browse/HDFS-17069
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: dfs, documentation
> Affects Versions: 3.3.6
> Environment: Linux version 4.15.0-142-generic
> (buildd@lgw01-amd64-039) (gcc version 5.4.0 20160609 (Ubuntu
> 5.4.0-6ubuntu1~16.04.12))
> java version "1.8.0_162"
> Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
> Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
> Reporter: ECFuzz
> Priority: Major
>
> My hadoop version is 3.3.6, and I use the Pseudo-Distributed Operation.
> `core-site.xml` like below.
> {quote}```shell
> <configuration>
> <property>
> <name>fs.defaultFS</name>
> <value>hdfs://localhost:9000</value>
> </property>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/home/hadoop/Mutil_Component/tmp</value>
> </property>
>
> </configuration>
> ```
> {quote}
> `hdfs-site.xml` like below.
> ```shell
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.blocksize</name>
> <value>128k</value>
> </property>
>
> </configuration>
> ```
>
> And then format the namenode, and start the hdfs.
> ```shell
> hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
> bin/hdfs namenode -format
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx(many info)
> hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
> sbin/start-dfs.sh
> Starting namenodes on [localhost]
> Starting datanodes
> Starting secondary namenodes [hadoop-Standard-PC-i440FX-PIIX-1996]
> ```
> Finally, use dfs to put a file. Then I get the message which means 128k is
> less than 1M.
> ```shell
> hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
> bin/hdfs dfs -mkdir -p /user/hadoop
> hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
> bin/hdfs dfs -mkdir input
> hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$
> bin/hdfs dfs -put etc/hadoop/hdfs-site.xml input
> put: Specified block size is less than configured minimum value
> (dfs.namenode.fs-limits.min-block-size): 131072 < 1048576
> ```
>
> But I find that in the document, dfs.blocksize can be set like 128k and other
> values in `hdfs-default.xml` .
> ```shell
> The default block size for new files, in bytes. You can use the following
> suffix (case insensitive): k(kilo), m(mega), g(giga), t(tera), p(peta),
> e(exa) to specify the size (such as 128k, 512m, 1g, etc.), Or provide
> complete size in bytes (such as 134217728 for 128 MB).
> ```
> So, should there be some issues with the documents here？Or should notice user
> to set this configuration to be lager than 1M?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
