[
https://issues.apache.org/jira/browse/HDFS-10867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419934#comment-16419934
]
Chris Douglas commented on HDFS-10867:
--------------------------------------
bq. Don't forget about legacy negative block ids that already aren't compatible
with EC assumptions. You can't rob the upper bits of a block id.
*nod* we saw HDFS-13350. Each partition increases the chances of a collision
with a legacy ID (HDFS-898, particularly [~szetszwo]'s analysis). I'd stop
short of considering partitioning as "robbing" the block ID namespace, though.
We need a fix for EC blocks, and if our solution doesn't also cover this case,
then it's probably incomplete. Since these features are only in 3.x, I would be
comfortable adding a step to the upgrade guide that instructs users to run fsck
and re-copy files with legacy block IDs (or never use these features). Of
course, a tool or comprehensive solution would be better.
When crossing a major version, we can stop supporting some cases. Requiring
work from users to eliminate legacy block IDs as an ongoing concern... seems
like an OK tradeoff, to me.
> [PROVIDED Phase 2] Block Bit Field Allocation of Provided Storage
> -----------------------------------------------------------------
>
> Key: HDFS-10867
> URL: https://issues.apache.org/jira/browse/HDFS-10867
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs
> Reporter: Ewan Higgs
> Priority: Major
> Attachments: Block Bit Field Allocation of Provided Storage.pdf
>
>
> We wish to design and implement the following related features for provided
> storage:
> # Dynamic mounting of provided storage within a Namenode (mount, unmount)
> # Mount multiple provided storage systems on a single Namenode.
> # Support updates to the provided storage system without having to regenerate
> an fsimg.
> A mount in the namespace addresses a corresponding set of block data. When
> unmounted, any block data associated with the mount becomes invalid and
> (eventually) unaddressable in HDFS. As with erasure-coded blocks, efficient
> unmounting requires that all blocks with that attribute be identifiable by
> the block management layer
> In this subtask, we focus on changes and conventions to the block management
> layer. Namespace operations are covered in a separate subtask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
