[
https://issues.apache.org/jira/browse/HDFS-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16258852#comment-16258852
]
caixiaofeng commented on HDFS-2535:
-----------------------------------
mark it here.
> A Model for Data Durability
> ---------------------------
>
> Key: HDFS-2535
> URL: https://issues.apache.org/jira/browse/HDFS-2535
> Project: Hadoop HDFS
> Issue Type: Task
> Components: documentation
> Reporter: Robert Chansler
> Assignee: Robert Chansler
> Attachments: LosingBlocks.xlsx
>
>
> This is a statistical model considering data durability in HDFS in the
> presence of Data Node failures. The attached spreadsheet considers the
> probability of losing a block with three replicas in the case of uncorrelated
> failures of DNs. Also included is a section that looks at the consequences of
> simultaneous failures.
> The model parameters reflect experience at Yahoo with a large cluster. But it
> is easy to change the parameters in the spreadsheet. Number of replicas is
> not a easily adjusted parameter. And while published reports (the Google
> papers) suggest that node failures are not really uncorrelated, this does
> give some practical insight into HDFS durability.
> I and others have quoted from this work in the past. I thought it good to
> make the details conveniently available.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
