On 2001-02-07 05:20 -0800, Peter Wemm [EMAIL PROTECTED] wrote:
[ Follow-ups to the FreeBSD-Audit mail list only, please ... ]
Poul-Henning Kamp wrote:
In message [EMAIL PROTECTED], Peter Wemm writes:
So that fsck(8) can see what mode the FS *was* mounted in last time. That
bears no
Stefan Esser wrote:
On 2001-02-07 05:20 -0800, Peter Wemm [EMAIL PROTECTED] wrote:
[ Follow-ups to the FreeBSD-Audit mail list only, please ... ]
Poul-Henning Kamp wrote:
In message [EMAIL PROTECTED], Peter Wemm writes
:
So that fsck(8) can see what mode the FS *was* mounted in last
Matt Dillon writes:
:Matt Dillon wrote:
: Yes. In general softupdates will make the entire filesystem safer.
:Does it make sense to use softupdates on file systems like / and
:/usr which have little file creation/removal?
I have had softupdates turned on for all of my mount points
: My recommendation is to turn softupdates on for everything you have,
: and for us to make it a newfs default as well. At least in -stable.
:You use softupdates turned on for all of your ufs.
:Understand.
:What is the reason to use softupdates for file system
:with only atime updates on
Matt Dillon wrote:
Well, after a long conversation with Mr Bernstein and Kirk it turns out
that all my blathering about a normal FFS mount being easily corruptable
due to a crash occuring during heavy disk I/O (e.g. from qmail) is so
much smoke.
The fsync()/rename()
:I do. Is it safe there as well (from your point of view)?
:
:--
:Andre
Yes. In general softupdates will make the entire filesystem safer.
The commit sequencing doesn't match what qmail expects, but there
are so many fsyncs going on that the absolute worse that can happen
in a
Matt Dillon wrote:
Yes. In general softupdates will make the entire filesystem safer.
Does it make sense to use softupdates on file systems like / and
/usr which have little file creation/removal?
Greg
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in
:
:Matt Dillon wrote:
:
: Yes. In general softupdates will make the entire filesystem safer.
:
:Does it make sense to use softupdates on file systems like / and
:/usr which have little file creation/removal?
:
:Greg
I have had softupdates turned on for all of my mount points for over
Greg Black [EMAIL PROTECTED] wrote:
Tony Finch wrote:
Why not just use rename(2)? To protect against the new filename
already existing?
Why not just read the man page for rename(2) before making
suggestions?
I did. I'm glad I was right that it's deleting the destination that is
the problem.
: already existing?
:
:Why not just read the man page for rename(2) before making
:suggestions?
:
:I did. I'm glad I was right that it's deleting the destination that is
:the problem. I would have thought it would be easy to be sure that
:spool filenames are unique, but OTOH I guess that's not
Well, after a long conversation with Mr Bernstein and Kirk it turns out
that all my blathering about a normal FFS mount being easily corruptable
due to a crash occuring during heavy disk I/O (e.g. from qmail) is so
much smoke.
The fsync()/rename() combination that QMail does
In message [EMAIL PROTECTED], Greg Black writes:
Matt Dillon wrote:
And, I would say, that for any mailer creating and deleting files in
a spool directory at a high rate, *ONLY* a filesystem with softupdates
turned on or a journaling filesystem such as XFS or ReiserFS can be
Poul-Henning Kamp wrote:
In message [EMAIL PROTECTED], Greg Black writes:
Matt Dillon wrote:
It seems to me that you're saying that softupdates is now the
recommended way to go -- so why does 4.2-Release still have the
dire warnings in /sys/ufs/ffs/README.softupdates? Is that file
Andre Oppermann [EMAIL PROTECTED] wrote:
Qmail has a couple of directories for the different states a queued
message goes through. The whole queue structure is required to be on
the same partition/disk. After the completing of each step in the queue
it is moved through the use of link() and then
Tony Finch wrote:
Why not just use rename(2)? To protect against the new filename
already existing?
Why not just read the man page for rename(2) before making
suggestions?
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
In message [EMAIL PROTECTED], Maxime Henrion writes:
What do you think of what NetBSD implemented ? softupdates is now enabled via
a mount option. This seems cleaner than the tunefs -n enable thing.
I have never understood why it was a tunefs thing...
--
Poul-Henning Kamp | UNIX since
Poul-Henning Kamp wrote:
In message [EMAIL PROTECTED], Maxime Henrion writes:
What do you think of what NetBSD implemented ? softupdates is now enabled vi
a
a mount option. This seems cleaner than the tunefs -n enable thing.
I have never understood why it was a tunefs thing...
So
In message [EMAIL PROTECTED], Peter Wemm writes:
Poul-Henning Kamp wrote:
In message [EMAIL PROTECTED], Maxime Henrion writes:
What do you think of what NetBSD implemented ? softupdates is now enabled vi
a
a mount option. This seems cleaner than the tunefs -n enable thing.
I have
At 21:25 07/02/01 +1000, Greg Black wrote:
Tony Finch wrote:
Why not just use rename(2)? To protect against the new filename
already existing?
Why not just read the man page for rename(2) before making
suggestions?
I find nothing convincing in the manpage. Could you please tell
what I am
mouss wrote:
At 21:25 07/02/01 +1000, Greg Black wrote:
Tony Finch wrote:
Why not just use rename(2)? To protect against the new filename
already existing?
Why not just read the man page for rename(2) before making
suggestions?
I find nothing convincing in the manpage. Could you
* Greg Black [EMAIL PROTECTED] [010207 13:05] wrote:
mouss wrote:
At 21:25 07/02/01 +1000, Greg Black wrote:
Tony Finch wrote:
Why not just use rename(2)? To protect against the new filename
already existing?
Why not just read the man page for rename(2) before making
Alfred Perlstein wrote:
* Greg Black [EMAIL PROTECTED] [010207 13:05] wrote:
mouss wrote:
At 21:25 07/02/01 +1000, Greg Black wrote:
Tony Finch wrote:
Why not just use rename(2)? To protect against the new filename
already existing?
Why not just read the man page
* Greg Black [EMAIL PROTECTED] [010207 17:33] wrote:
Alfred Perlstein wrote:
* Greg Black [EMAIL PROTECTED] [010207 13:05] wrote:
mouss wrote:
At 21:25 07/02/01 +1000, Greg Black wrote:
Tony Finch wrote:
Why not just use rename(2)? To protect against the new
:
:In message [EMAIL PROTECTED],
:Charles Randall writes:
:The qmail FAQ specifically recommends against soft updates for the mail
:queue.
:
:http://cr.yp.to/qmail/faq/reliability.html#filesystems
:
:Is this incorrect?
:
:
:It seems to indicate that qmail doesn't use fsync(2) as much as it
On Tue, 6 Feb 2001, Poul-Henning Kamp wrote:
In message [EMAIL PROTECTED],
Charles Randall writes:
The qmail FAQ specifically recommends against soft updates for the mail
queue.
http://cr.yp.to/qmail/faq/reliability.html#filesystems
Is this incorrect?
It seems to indicate that qmail
On Tue, 6 Feb 2001, Rik van Riel wrote:
The system call used to guarantee this is fsync (and friends?);
if qmail doesn't use it but makes assumptions that aren't true
on any decent OS out there ...
regards,
Rik
Well, the various qmail programs do seem to fsync (though I'm not sure if
In message [EMAIL PROTECTED], Matt Dillon writes:
:
:In message [EMAIL PROTECTED],
Charles Randall writes:
:The qmail FAQ specifically recommends against soft updates for the mail
:queue.
:
:http://cr.yp.to/qmail/faq/reliability.html#filesystems
:
:Is this incorrect?
:
:
:It seems to indicate
:softupdates disk may wind up unwinding 'more' of the last few moments
:worth of operations then a normal filesystem would. And, I might add,
:Reiser is the same way.
:
:The only way to guarentee that file data is written to disk, with any
:filesystem no matter how it is
:Well, the various qmail programs do seem to fsync (though I'm not sure if
:it's in the right places.) In any case, this link seems to throw some
:light on the situation:
:
:ftp://elektroni.ee.tut.fi/pub/qmail_linux_metadata_message
:
:Now, I have no clue if this is correct or not, but the core
On Tue, 6 Feb 2001, Mike Silbersack wrote:
On Tue, 6 Feb 2001, Rik van Riel wrote:
The system call used to guarantee this is fsync (and friends?);
if qmail doesn't use it but makes assumptions that aren't true
on any decent OS out there ...
Well, the various qmail programs do seem to
On Tue, 6 Feb 2001, Matt Dillon wrote:
QMail's FAQ is totally incorrect. No major filesystem -- be it
FFS, EX2FS, Reiser, FFS+Softupdates, guarentees that when you
write() and close() a file that the file will then survive a disk
crash. All these filesystems guarentee is
:with fsync(), so softupdates is not going to be too much worse then
:other FS's.
:
:Actually, if you don't use fsync you do loose more work with
:softupdates than if you use plain UFS.
:
:Softupdates can delay directory updates which plain UFS will runs
:synchronously, and consequently
In message [EMAIL PROTECTED], Matt Dillon writes:
:softupdates disk may wind up unwinding 'more' of the last few moments
:worth of operations then a normal filesystem would. And, I might add,
:Reiser is the same way.
:
:The only way to guarentee that file data is written to disk,
Mike Silbersack wrote:
On Tue, 6 Feb 2001, Rik van Riel wrote:
The system call used to guarantee this is fsync (and friends?);
if qmail doesn't use it but makes assumptions that aren't true
on any decent OS out there ...
regards,
Rik
Well, the various qmail programs do seem
:kirk said (but I have not completely checked it) that if you fsync a file,
:it will effectively fsync all the way back to the root of the filesystem.
:
:I don't know how true this is, but cerainly the inode is updated before
:fsync returns. I cannot tell if any directory entries pointing at
: consistent state. Softupdates is considerably better at guarenteeing
: this consistency (as is something like Reiser), but if you crash a
: softupdates disk may wind up unwinding 'more' of the last few moments
: worth of operations then a normal filesystem would. And, I might
Poul-Henning Kamp wrote:
In message [EMAIL PROTECTED],
Charles Randall writes:
The qmail FAQ specifically recommends against soft updates for the mail
queue.
http://cr.yp.to/qmail/faq/reliability.html#filesystems
Is this incorrect?
It seems to indicate that qmail doesn't use
On Tue, 6 Feb 2001, Matt Dillon wrote:
:Reiserfs and ext3 have write-ahead logs and, AFAIK, fsync()
:will not return until there is a commit point in the log.
:
:This means that fsync() will guarantee that the transactions
:won't be unwound (unless I've overlooked some weird special
:case
:Poul-Henning Kamp wrote:
:
: In message [EMAIL PROTECTED],
:Charles Randall writes:
: The qmail FAQ specifically recommends against soft updates for the mail
: queue.
:
: http://cr.yp.to/qmail/faq/reliability.html#filesystems
:
: Is this incorrect?
:
:
: It seems to indicate that qmail
On Tue, 6 Feb 2001, Matt Dillon wrote:
I did a quick search of the qmail site but couldn't find an email
address to report the FAQ issue to. If QMail calls fsync() in a
reasonable manner, then softupdates is perfectly safe and the QMail
FAQ needs to be updated to recommend
Rik van Riel wrote:
On Tue, 6 Feb 2001, Poul-Henning Kamp wrote:
In message [EMAIL PROTECTED],
Charles Randall writes:
The qmail FAQ specifically recommends against soft updates for the mail
queue.
http://cr.yp.to/qmail/faq/reliability.html#filesystems
Is this incorrect?
:On Tue, 6 Feb 2001, Matt Dillon wrote:
:
: I did a quick search of the qmail site but couldn't find an email
: address to report the FAQ issue to. If QMail calls fsync() in a
: reasonable manner, then softupdates is perfectly safe and the QMail
: FAQ needs to be updated to
Poul-Henning Kamp wrote:
In message [EMAIL PROTECTED], Matt Dillon writes:
:
:In message [EMAIL PROTECTED],
Charles Randall writes:
:The qmail FAQ specifically recommends against soft updates for the mail
:queue.
:
:http://cr.yp.to/qmail/faq/reliability.html#filesystems
:
:Is this
Matt Dillon wrote:
:softupdates disk may wind up unwinding 'more' of the last few moments
:worth of operations then a normal filesystem would. And, I might add,
:Reiser is the same way.
:
:The only way to guarentee that file data is written to disk, with any
:
Rik van Riel wrote:
On Tue, 6 Feb 2001, Mike Silbersack wrote:
On Tue, 6 Feb 2001, Rik van Riel wrote:
The system call used to guarantee this is fsync (and friends?);
if qmail doesn't use it but makes assumptions that aren't true
on any decent OS out there ...
Well, the
Matt Dillon wrote:
:Well, the various qmail programs do seem to fsync (though I'm not sure if
:it's in the right places.) In any case, this link seems to throw some
:light on the situation:
:
:ftp://elektroni.ee.tut.fi/pub/qmail_linux_metadata_message
:
:Now, I have no clue if this is
Rik van Riel wrote:
On Tue, 6 Feb 2001, Matt Dillon wrote:
QMail's FAQ is totally incorrect. No major filesystem -- be it
FFS, EX2FS, Reiser, FFS+Softupdates, guarentees that when you
write() and close() a file that the file will then survive a disk
crash. All these
On Tue, 6 Feb 2001, Andre Oppermann wrote:
Date: Tue, 06 Feb 2001 20:32:07 +0100
From: Andre Oppermann [EMAIL PROTECTED]
NAME
link - make a hard file link
DESCRIPTION
The link() function call atomically creates the specified directory
entry
On Tue, 6 Feb 2001, Andre Oppermann wrote:
Qmail depends on ordered-metadata updates (Terry! :-). That
means if you issue a link() to the new place and a unlink() in
the old place it should guarantee that the link() happens
*BEFORE* the unlink().
As it is, I can only recommend people to
Mike Silbersack wrote:
On Tue, 6 Feb 2001, Matt Dillon wrote:
I did a quick search of the qmail site but couldn't find an email
address to report the FAQ issue to. If QMail calls fsync() in a
reasonable manner, then softupdates is perfectly safe and the QMail
FAQ
: Pre-softupdate BSD semantics, apparently. Doesn't sound like
: the smartest thing to do when you want a reliable MTA...
:
:This description is not entirely right.
:
:Qmail depends on ordered-metadata updates (Terry! :-). That means
:if you issue a link() to the new place and a unlink() in the
On Tue, 6 Feb 2001, Andre Oppermann wrote:
Rik van Riel wrote:
On Tue, 6 Feb 2001, Matt Dillon wrote:
Reiserfs and ext3 have write-ahead logs and, AFAIK, fsync()
will not return until there is a commit point in the log.
Also FFS/UFS will not return before the file and directory entry
Matt Dillon wrote:
:On Tue, 6 Feb 2001, Matt Dillon wrote:
:
: I did a quick search of the qmail site but couldn't find an email
: address to report the FAQ issue to. If QMail calls fsync() in a
: reasonable manner, then softupdates is perfectly safe and the QMail
: FAQ
Rik van Riel wrote:
On Tue, 6 Feb 2001, Andre Oppermann wrote:
Qmail depends on ordered-metadata updates (Terry! :-). That
means if you issue a link() to the new place and a unlink() in
the old place it should guarantee that the link() happens
*BEFORE* the unlink().
As it is, I
:
:This information is in fact correct. Have a look at the FreeBSD link(2)
:man page:
:
:LINK(2) FreeBSD System Calls Manual
:LINK(2)
Andre, I think there *might* be a dozen people in the world that
understand UFS/FFS better then I do, but none of them
Since I'm still being cc'd on this... :)
* Matt Dillon [EMAIL PROTECTED] [010206 11:56] wrote:
:
:This information is in fact correct. Have a look at the FreeBSD link(2)
:man page:
:
:LINK(2) FreeBSD System Calls Manual
:LINK(2)
Andre, I think
On Tue, Feb 06, 2001 at 10:59:09AM -0800, Matt Dillon wrote:
I did a quick search of the qmail site but couldn't find an email
address to report the FAQ issue to.
Try sending mail to [EMAIL PROTECTED]
--
Jos Backus _/ _/_/_/"Modularity is not a hack."
Matt Dillon wrote:
: Pre-softupdate BSD semantics, apparently. Doesn't sound like
: the smartest thing to do when you want a reliable MTA...
:
:This description is not entirely right.
:
:Qmail depends on ordered-metadata updates (Terry! :-). That means
:if you issue a link() to the new
Rik van Riel wrote:
On Tue, 6 Feb 2001, Andre Oppermann wrote:
Rik van Riel wrote:
On Tue, 6 Feb 2001, Matt Dillon wrote:
Reiserfs and ext3 have write-ahead logs and, AFAIK, fsync()
will not return until there is a commit point in the log.
Also FFS/UFS will not return before
* Andre Oppermann [EMAIL PROTECTED] [010206 12:07] wrote:
Yes, my understanding of the meaning of "ordered meta-date update" as
I have grasped it from Terry's rants in the past years is not that all
meta-data updates on a filesystem have to be done one-after-the-other
but ordered in respect
: banging on unrelated areas of the filesystem in parallel, and an
: fsync() of one descriptor would have to wait for the entire filesystem
: to reach a synchronization point to guarentee metadata update ordering.
: This creates a serious scaleability issue within a filesystem!
:
Matt Dillon wrote:
:
:This information is in fact correct. Have a look at the FreeBSD link(2)
:man page:
:
:LINK(2) FreeBSD System Calls Manual
:LINK(2)
Andre, I think there *might* be a dozen people in the world that
understand UFS/FFS better then I do, but
On Tue, 6 Feb 2001, Andre Oppermann wrote:
But please answer me one question: Is the link() call atomically
in FFS/UFS w or w/o softupdates? Meaning when the call returns
the meta- data is written to stable storage like with fsync()?
Since when does `atomic' equal `synchronous' ?
regards,
Rik van Riel wrote:
On Tue, 6 Feb 2001, Andre Oppermann wrote:
But please answer me one question: Is the link() call atomically
in FFS/UFS w or w/o softupdates? Meaning when the call returns
the meta- data is written to stable storage like with fsync()?
Since when does `atomic'
* Matt Dillon [EMAIL PROTECTED] [010206 12:19] wrote:
: banging on unrelated areas of the filesystem in parallel, and an
: fsync() of one descriptor would have to wait for the entire filesystem
: to reach a synchronization point to guarentee metadata update ordering.
: This
Alfred Perlstein wrote:
* Andre Oppermann [EMAIL PROTECTED] [010206 12:07] wrote:
Yes, my understanding of the meaning of "ordered meta-date update" as
I have grasped it from Terry's rants in the past years is not that all
meta-data updates on a filesystem have to be done
On Tue, 6 Feb 2001, Andre Oppermann wrote:
Rik van Riel wrote:
On Tue, 6 Feb 2001, Andre Oppermann wrote:
But please answer me one question: Is the link() call atomically
in FFS/UFS w or w/o softupdates? Meaning when the call returns
the meta- data is written to stable storage like
* Andre Oppermann [EMAIL PROTECTED] [010206 12:30] wrote:
Rik van Riel wrote:
On Tue, 6 Feb 2001, Andre Oppermann wrote:
But please answer me one question: Is the link() call atomically
in FFS/UFS w or w/o softupdates? Meaning when the call returns
the meta- data is written to
-On [20010206 20:25], Andre Oppermann ([EMAIL PROTECTED]) wrote:
... provided that qmail calls fsync(2).
$ cd qmail-ldap/
$ grep fsync * | wc -l
21
Of course that says nothing if the fsync()'s are not placed at strategic
places.
fsync();
fsync();
fsync();
.
.
.
fsync();
And _then_ the
Alfred Perlstein wrote:
* Andre Oppermann [EMAIL PROTECTED] [010206 12:30] wrote:
Rik van Riel wrote:
On Tue, 6 Feb 2001, Andre Oppermann wrote:
But please answer me one question: Is the link() call atomically
in FFS/UFS w or w/o softupdates? Meaning when the call returns
* Andre Oppermann [EMAIL PROTECTED] [010206 12:33] wrote:
Alfred Perlstein wrote:
* Andre Oppermann [EMAIL PROTECTED] [010206 12:07] wrote:
Yes, my understanding of the meaning of "ordered meta-date update" as
I have grasped it from Terry's rants in the past years is not that all
Alfred Perlstein wrote:
* Andre Oppermann [EMAIL PROTECTED] [010206 12:33] wrote:
Alfred Perlstein wrote:
* Andre Oppermann [EMAIL PROTECTED] [010206 12:07] wrote:
Yes, my understanding of the meaning of "ordered meta-date update" as
I have grasped it from Terry's rants in
Alfred Perlstein wrote:
Does sendmail even use fsync()?
It better. :)
since the last time I know of, Kirk and Eric lived in the same building
it would seem likely that Eric would know to do that..
--
__--_|\ Julian Elischer
/ \ [EMAIL PROTECTED]
( OZ)
* Andre Oppermann [EMAIL PROTECTED] [010206 12:58] wrote:
Alfred Perlstein wrote:
Basically, you want a fsync right before the IPC. This should
bring the metadata up to date with what's in-core and you should
then be safe when you reply with your 250 accepted message.
Like this
Alfred Perlstein wrote:
* Andre Oppermann [EMAIL PROTECTED] [010206 12:58] wrote:
Alfred Perlstein wrote:
Basically, you want a fsync right before the IPC. This should
bring the metadata up to date with what's in-core and you should
then be safe when you reply with your 250
Andre Oppermann wrote:
Since when does `atomic' equal `synchronous' ?
Because otherwise it would not be atomically, would it?
I am loath to add to this bloated thread, but... atomic and durable
aren't the same thing. This is why A.C.I.D. semantics contain both A
D. The atomicity
On Tue, Feb 06, 2001 at 12:13:57PM -0800, Alfred Perlstein wrote:
* Andre Oppermann [EMAIL PROTECTED] [010206 12:07] wrote:
Does sendmail even use fsync()?
It better. :)
Quick grep of the sendmail sources shows most of the six fsync
calls protected by a flag (SuperSafe or nofsync ). I
Matt Dillon wrote:
And, I would say, that for any mailer creating and deleting files in
a spool directory at a high rate, *ONLY* a filesystem with softupdates
turned on or a journaling filesystem such as XFS or ReiserFS can be
considered crash-surviveable. Synchronous
78 matches
Mail list logo
