[ https://issues.apache.org/jira/browse/HDFS-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16258852#comment-16258852 ]
caixiaofeng commented on HDFS-2535: ----------------------------------- mark it here. > A Model for Data Durability > --------------------------- > > Key: HDFS-2535 > URL: https://issues.apache.org/jira/browse/HDFS-2535 > Project: Hadoop HDFS > Issue Type: Task > Components: documentation > Reporter: Robert Chansler > Assignee: Robert Chansler > Attachments: LosingBlocks.xlsx > > > This is a statistical model considering data durability in HDFS in the > presence of Data Node failures. The attached spreadsheet considers the > probability of losing a block with three replicas in the case of uncorrelated > failures of DNs. Also included is a section that looks at the consequences of > simultaneous failures. > The model parameters reflect experience at Yahoo with a large cluster. But it > is easy to change the parameters in the spreadsheet. Number of replicas is > not a easily adjusted parameter. And while published reports (the Google > papers) suggest that node failures are not really uncorrelated, this does > give some practical insight into HDFS durability. > I and others have quoted from this work in the past. I thought it good to > make the details conveniently available. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org