[
https://issues.apache.org/jira/browse/HADOOP-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HADOOP-12981:
-------------------------------------
Description:
It seems all properties defined in {{S3NativeFileSystemConfigKeys}} are not
used. The current s3native properties are all prefixed by {{fs.s3n}}, so this
is likely not used currently. Additionally, core-default.xml has the
description of these unused properties:
{noformat}
<!-- s3native File System -->
<property>
<name>s3native.stream-buffer-size</name>
<value>4096</value>
<description>The size of buffer to stream files.
The size of this buffer should probably be a multiple of hardware
page size (4096 on Intel x86), and it determines how much data is
buffered during read and write operations.</description>
</property>
<property>
<name>s3native.bytes-per-checksum</name>
<value>512</value>
<description>The number of bytes per checksum. Must not be larger than
s3native.stream-buffer-size</description>
</property>
<property>
<name>s3native.client-write-packet-size</name>
<value>65536</value>
<description>Packet size for clients to write</description>
</property>
<property>
<name>s3native.blocksize</name>
<value>67108864</value>
<description>Block size</description>
</property>
<property>
<name>s3native.replication</name>
<value>3</value>
<description>Replication factor</description>
</property>
{noformat}
I think they should be removed (or deprecated) to avoid confusion if these
properties are defunct.
was:
It seems all properties defined in {{S3NativeFileSystemConfigKeys}} are not
used. Additionally, core-default.xml has the description of these unused
properties:
{noformat}
<!-- s3native File System -->
<property>
<name>s3native.stream-buffer-size</name>
<value>4096</value>
<description>The size of buffer to stream files.
The size of this buffer should probably be a multiple of hardware
page size (4096 on Intel x86), and it determines how much data is
buffered during read and write operations.</description>
</property>
<property>
<name>s3native.bytes-per-checksum</name>
<value>512</value>
<description>The number of bytes per checksum. Must not be larger than
s3native.stream-buffer-size</description>
</property>
<property>
<name>s3native.client-write-packet-size</name>
<value>65536</value>
<description>Packet size for clients to write</description>
</property>
<property>
<name>s3native.blocksize</name>
<value>67108864</value>
<description>Block size</description>
</property>
<property>
<name>s3native.replication</name>
<value>3</value>
<description>Replication factor</description>
</property>
{noformat}
I think they should be removed (or deprecated) to avoid confusion if these
properties are defunct.
> Remove/deprecate s3native properties from S3NativeFileSystemConfigKeys and
> core-default.xml
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-12981
> URL: https://issues.apache.org/jira/browse/HADOOP-12981
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools
> Affects Versions: 3.0.0
> Reporter: Wei-Chiu Chuang
> Priority: Minor
> Labels: aws
>
> It seems all properties defined in {{S3NativeFileSystemConfigKeys}} are not
> used. The current s3native properties are all prefixed by {{fs.s3n}}, so this
> is likely not used currently. Additionally, core-default.xml has the
> description of these unused properties:
> {noformat}
> <!-- s3native File System -->
> <property>
> <name>s3native.stream-buffer-size</name>
> <value>4096</value>
> <description>The size of buffer to stream files.
> The size of this buffer should probably be a multiple of hardware
> page size (4096 on Intel x86), and it determines how much data is
> buffered during read and write operations.</description>
> </property>
> <property>
> <name>s3native.bytes-per-checksum</name>
> <value>512</value>
> <description>The number of bytes per checksum. Must not be larger than
> s3native.stream-buffer-size</description>
> </property>
> <property>
> <name>s3native.client-write-packet-size</name>
> <value>65536</value>
> <description>Packet size for clients to write</description>
> </property>
> <property>
> <name>s3native.blocksize</name>
> <value>67108864</value>
> <description>Block size</description>
> </property>
> <property>
> <name>s3native.replication</name>
> <value>3</value>
> <description>Replication factor</description>
> </property>
> {noformat}
> I think they should be removed (or deprecated) to avoid confusion if these
> properties are defunct.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
