Re: Some very basic questions

2008-10-27 Thread Stephan von Krawczynski
On Wed, 22 Oct 2008 16:35:55 +0200 dbz [EMAIL PROTECTED] wrote: concerning this discussion, I'd like to put up some requests which strongly oppose to those brought up initially: - if you run into an error in the fs structure or any IO error that prevents you from bringing the fs into a

Re[2]: Some very basic questions

2008-10-27 Thread sftf
In my opinon, the whole thing comes up from the idea of using cheap hardware and out-of-the-box configurations to keep promises of reliability and availability which are not realistic. There is a reason why there are more expensive HDDs, RAIDs, SANs with volume mirroring, multipathing and so

Re: Some very basic questions

2008-10-22 Thread Ric Wheeler
Eric Anopolsky wrote: On Tue, 2008-10-21 at 18:18 -0400, Ric Wheeler wrote: Eric Anopolsky wrote: On Tue, 2008-10-21 at 09:59 -0400, Chris Mason wrote: - power loss at any time must not corrupt the fs (atomic fs modification) (new-data loss is acceptable)

Re: Some very basic questions

2008-10-22 Thread Stephan von Krawczynski
On Tue, 21 Oct 2008 13:15:13 -0400 Christoph Hellwig [EMAIL PROTECTED] wrote: On Tue, Oct 21, 2008 at 07:01:36PM +0200, Stephan von Krawczynski wrote: Sure, but what you say only reflects the ideal world. On a file service, you never have that. In fact you do not even have good control about

Re: Some very basic questions

2008-10-22 Thread Stephan von Krawczynski
On Tue, 21 Oct 2008 18:09:40 +0200 Andi Kleen [EMAIL PROTECTED] wrote: While that's true today, I'm not sure it has to be true always. I always thought traditional fsck user interfaces were a UI desaster and could be done much better with some simple tweaks. [...] You are completely right.

Re: Some very basic questions

2008-10-22 Thread Stephan von Krawczynski
On Tue, 21 Oct 2008 18:59:26 +0200 Andi Kleen [EMAIL PROTECTED] wrote: Stephan von Krawczynski [EMAIL PROTECTED] writes: Yes, we hear and say that all the time, name one linux fs doing it, please. ext[234] support it to some extent. It has some limitations (especially when the files are

Re: Some very basic questions

2008-10-22 Thread Stephan von Krawczynski
On Tue, 21 Oct 2008 11:34:20 -0400 jim owens [EMAIL PROTECTED] wrote: Hearing what user's think they want is always good, but... Stephan von Krawczynski wrote: thanks for your feedback. Understand minimum requirement as minimum requirement to drop the current installation and migrate

Re: Some very basic questions

2008-10-22 Thread Avi Kivity
Stephan von Krawczynski wrote: - filesystem autodetects, isolates, and (possibly) repairs errors - online scan, check, repair filesystem tool initiated by admin - Reliability so high that they never run that check-and-fix tool That is _wrong_ (to a certain extent). You _want to

Re: Some very basic questions

2008-10-22 Thread Stephan von Krawczynski
On Tue, 21 Oct 2008 13:49:43 -0400 Chris Mason [EMAIL PROTECTED] wrote: On Tue, 2008-10-21 at 18:27 +0200, Stephan von Krawczynski wrote: 2. general requirements - fs errors without file/dir names are useless - errors in parts of the fs are no reason for a fs to go offline

Re: Some very basic questions

2008-10-22 Thread Ric Wheeler
Tejun Heo wrote: Ric Wheeler wrote: The cache flush command for ATA devices will block and wait until all of the device's write cache has been written back. What I assume Tejun was referring to here is that some IO might have been written out to the device and an error happened when the

Re: Some very basic questions

2008-10-22 Thread Ric Wheeler
Avi Kivity wrote: Stephan von Krawczynski wrote: - filesystem autodetects, isolates, and (possibly) repairs errors - online scan, check, repair filesystem tool initiated by admin - Reliability so high that they never run that check-and-fix tool That is _wrong_ (to a certain

Re: Some very basic questions

2008-10-22 Thread Chris Mason
On Wed, 2008-10-22 at 14:27 +0200, Stephan von Krawczynski wrote: On Tue, 21 Oct 2008 13:31:37 -0400 Ric Wheeler [EMAIL PROTECTED] wrote: [...] If you have remapped a big chunk of the sectors (say more than 10%), you should grab the data off the disk asap and replace it. Worry less

Re: Some very basic questions

2008-10-22 Thread Chris Mason
On Wed, 2008-10-22 at 22:15 +0900, Tejun Heo wrote: Ric Wheeler wrote: I think that we do handle a failure in the case that you outline above since the FS will be able to notice the error before it sends a commit down (and that commit is wrapped in the barrier flush calls). This is the

Re: Some very basic questions

2008-10-22 Thread Stephan von Krawczynski
On Wed, 22 Oct 2008 05:48:30 -0700 Jeff Schroeder [EMAIL PROTECTED] wrote: NFS is a good example for a fs that never got redesigned for modern world. I hope it will, but currently it's like Model T on a highway. You have a NFS server with clients. Your NFS server dies, your backup server

Re: Some very basic questions

2008-10-22 Thread Matthias Wächter
On 10/22/2008 3:50 PM, Chris Mason wrote: Let me reword my answer ;). The next write will always succeed unless the drive is out of remapping sectors. If the drive is out, it is only good for reads and holding down paper on your desk. I have a fairly new SATA disk with about 3000 hours of

Re: Some very basic questions

2008-10-22 Thread Ric Wheeler
Chris Mason wrote: On Wed, 2008-10-22 at 09:38 -0400, Ric Wheeler wrote: Chris Mason wrote: On Wed, 2008-10-22 at 22:15 +0900, Tejun Heo wrote: Ric Wheeler wrote: I think that we do handle a failure in the case that you outline above since the FS will be able

Re: Some very basic questions

2008-10-22 Thread Avi Kivity
jim owens wrote: Avi Kivity wrote: jim owens wrote: Remember that the device bandwidth is the limiter so even when each host has a dedicated path to the device (as in dual port SAS or FC), that 2nd host cuts the throughput by more than 1/2 with uncoordinated seeks and transfers. That's only

Re: Some very basic questions

2008-10-22 Thread Avi Kivity
Ric Wheeler wrote: One key is not to replace the drives too early - you often can recover significant amounts of data from a drive that is on its last legs. This can be useful even in RAID rebuilds since with today's enormous drive capacities, you might hit a latent error during the rebuild on

Re: Some very basic questions

2008-10-22 Thread dbz
concerning this discussion, I'd like to put up some requests which strongly oppose to those brought up initially: - if you run into an error in the fs structure or any IO error that prevents you from bringing the fs into a consistent state, please simply oops. If a user feels that

Re: Some very basic questions

2008-10-22 Thread Chris Mason
On Wed, 2008-10-22 at 16:32 +0200, Avi Kivity wrote: Ric Wheeler wrote: One key is not to replace the drives too early - you often can recover significant amounts of data from a drive that is on its last legs. This can be useful even in RAID rebuilds since with today's enormous drive

Re: Some very basic questions

2008-10-22 Thread Avi Kivity
Chris Mason wrote: You want to have spare capacity, enough for one or two (or fifteen) drives' worth of data. When a drive goes bad, you rebuild into the spare capacity you have. You want spare capacity that does not degrade your raid levels if you move the data onto it. In some

Re: Some very basic questions

2008-10-22 Thread jim owens
Ric Wheeler wrote: Matthias Wächter wrote: On 10/22/2008 3:50 PM, Chris Mason wrote: Let me reword my answer ;). The next write will always succeed unless the drive is out of remapping sectors. If the drive is out, it is only good for reads and holding down paper on your desk. I

Re: Some very basic questions

2008-10-22 Thread Ric Wheeler
Avi Kivity wrote: Ric Wheeler wrote: You want to have spare capacity, enough for one or two (or fifteen) drives' worth of data. When a drive goes bad, you rebuild into the spare capacity you have. That is a different model (and one that makes sense, we used that in Centera for object level

Re: Some very basic questions

2008-10-22 Thread Avi Kivity
Ric Wheeler wrote: I think that the btrfs plan is still to push more complicated RAID schemes off to MD (RAID6, etc) so this is an issue even with a JBOD. It will be interesting to map out the possible ways to use built in mirroring, etc vs the external RAID and actually measure the utilized

Re: Some very basic questions

2008-10-22 Thread jim owens
Michel Salim wrote: Though it would be nice to have a tool that would provide enough information to make a warranty claim -- does btrfs keep enough information for such a tool to be written? Failed device I/O (rather than bad checksums and other fs-specific error detections) should be logged

Re: Some very basic questions

2008-10-22 Thread Avi Kivity
Tejun Heo wrote: For most SATA drives, disabling write back cache seems to take high toll on write throughput. :-( I measured this yesterday. This is true for pure write workloads; for mixed read/write workloads the throughput decrease is negligible. As long as the error status is

Re: Some very basic questions

2008-10-21 Thread Chris Mason
On Tue, 2008-10-21 at 13:23 +0200, Stephan von Krawczynski wrote: Hello all, reading the list for a while it looks like all kinds of implementational topics are covered but no basic user requests or talks are going on. Since I have found no other list on vger covering these issues I choose

Re: Some very basic questions

2008-10-21 Thread Stephan von Krawczynski
On Tue, 21 Oct 2008 14:13:33 +0200 Andi Kleen [EMAIL PROTECTED] wrote: Stephan von Krawczynski [EMAIL PROTECTED] writes: reading the list for a while it looks like all kinds of implementational topics are covered but no basic user requests or talks are going on. Since I have found no

Re: Some very basic questions

2008-10-21 Thread Andi Kleen
Chris Mason [EMAIL PROTECTED] writes: Started interactively? I'm not entirely sure what that means, but in general when you ask the user a question about if/how to fix a corruption, they will have no idea what the correct answer is. While that's true today, I'm not sure it has to be true

Re: Some very basic questions

2008-10-21 Thread Stephan von Krawczynski
On Tue, 21 Oct 2008 09:20:16 -0400 jim owens [EMAIL PROTECTED] wrote: btrfs has many of the same goals... but they are goals not code so when you might see them is indeterminate. no big issue, my pension is 20 years away, I got time ;-) I believe these should not be in btrfs: Stephan von

Re: Some very basic questions

2008-10-21 Thread Ric Wheeler
Christoph Hellwig wrote: On Tue, Oct 21, 2008 at 07:01:36PM +0200, Stephan von Krawczynski wrote: Sure, but what you say only reflects the ideal world. On a file service, you never have that. In fact you do not even have good control about what is going on. Lets say you have a setup that

Re: Some very basic questions

2008-10-21 Thread calin
question is: if you had such an implementation, are there drawbacks expectable for the single-mount case? If not I'd vote for it because there are not really many alternatives on the market. As I understand it, the largest issue is in locking and boundaries. Two different systems could mount

Re: Some very basic questions

2008-10-21 Thread Chris Mason
On Tue, 2008-10-21 at 18:27 +0200, Stephan von Krawczynski wrote: 2. general requirements - fs errors without file/dir names are useless - errors in parts of the fs are no reason for a fs to go offline as a whole These two are in progress. Btrfs won't always be able to

Re: Some very basic questions

2008-10-21 Thread Eric Anopolsky
On Tue, 2008-10-21 at 18:18 -0400, Ric Wheeler wrote: Eric Anopolsky wrote: On Tue, 2008-10-21 at 09:59 -0400, Chris Mason wrote: - power loss at any time must not corrupt the fs (atomic fs modification) (new-data loss is acceptable) Done. Btrfs already uses