[
https://issues.apache.org/jira/browse/HDFS-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277548#comment-14277548
]
Jing Zhao commented on HDFS-3689:
---------------------------------
Sure. Currently the patch includes the following changes:
# Add a new append2 API which always appends new data to a new block, and thus
the previous last block becomes a block with variable length. Some unit tests
are added to make sure the appended file can still be read/pread by the current
DFSInputStream.
# Add support in DFSOutputStream to let clients specify when to allocate a new
block. The patch simply adds a new SyncFlag named END_BLOCK. Clients can call
hsync(END_BLOCK) to force the completion of the current block.
# Loose the restriction of concat. The current concat has the following
restrictions:
- The src files and the target file must be in the same directory and cannot be
empty
- The target file and all but the last src files cannot have partial block
- The src files and the target file must share the same replication factor and
preferred block size
- The src files and the target file cannot be in any snapshot
The current patch makes the following changes which I think needs further
discussion and confirmation:
- The src files and the target file do not need to be in the same directory
- The src files and the target file can have partial blocks
- The src/target files may have different preferred block size and replication
factor, and after the concat the target file keeps its original setting
- The src files still cannot be included in any snapshot (see HDFS-4529 for
details), but the target file can be in a snapshot
> Add support for variable length block
> -------------------------------------
>
> Key: HDFS-3689
> URL: https://issues.apache.org/jira/browse/HDFS-3689
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, hdfs-client, namenode
> Affects Versions: 3.0.0
> Reporter: Suresh Srinivas
> Assignee: Jing Zhao
> Attachments: HDFS-3689.000.patch, HDFS-3689.001.patch,
> HDFS-3689.002.patch, HDFS-3689.003.patch, HDFS-3689.003.patch,
> HDFS-3689.004.patch
>
>
> Currently HDFS supports fixed length blocks. Supporting variable length block
> will allow new use cases and features to be built on top of HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
