[
https://issues.apache.org/jira/browse/HDFS-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439715#comment-13439715
]
Robert Chansler commented on HDFS-2535:
---------------------------------------
At Hadoop Summit 2011: http://www.youtube.com/watch?v=zbycDpVWhp0
Also referenced in ;login: February 2012:
https://www.usenix.org/publications/login/february-2012/data-availability-and-durability-hadoop-distributed-file-system
> A Model for Data Durability
> ---------------------------
>
> Key: HDFS-2535
> URL: https://issues.apache.org/jira/browse/HDFS-2535
> Project: Hadoop HDFS
> Issue Type: Task
> Components: documentation
> Reporter: Robert Chansler
> Assignee: Robert Chansler
> Attachments: LosingBlocks.xlsx
>
>
> This is a statistical model considering data durability in HDFS in the
> presence of Data Node failures. The attached spreadsheet considers the
> probability of losing a block with three replicas in the case of uncorrelated
> failures of DNs. Also included is a section that looks at the consequences of
> simultaneous failures.
> The model parameters reflect experience at Yahoo with a large cluster. But it
> is easy to change the parameters in the spreadsheet. Number of replicas is
> not a easily adjusted parameter. And while published reports (the Google
> papers) suggest that node failures are not really uncorrelated, this does
> give some practical insight into HDFS durability.
> I and others have quoted from this work in the past. I thought it good to
> make the details conveniently available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
