Re: {a}sync updates (was Re: make install trick)

[Tony Finch]

I have noticed similarly odd behaviour from softupdates during heavy
IO load, where something is creating lots of little files or
directories and not much else is happening. Using `vmstat 1` I can see
that softupdates isn't very good at evening out the IO rate over time:
there's a roughly sinusoidal back-and-forth between frantic disk
thrashing (lots of TPS) and lots of syscall activity. Visible progress
is correspondingly uneven.

For example, the `vmstat 1` output while the script
for i in `jot 256`
do
for j in `jot 256`
do
mkdir -p $i/$j
done
done
is running is appended below. Behaviour is much smoother during an
`rm -r *` of the resulting directory tree.

Tony.
-- 
f.a.n.finch[EMAIL PROTECTED][EMAIL PROTECTED]
  Apache Software Foundation Member

 procs  memory page   disksfaults  cpu
 r b w avm   fre  flt  re  pi  po  fr  sr da0 fd0 pa0   in   sy  cs us sy id
 2 1 0 4183836 11172 3389   0   0   0 3188   0 273   0   0  512 6015 1210 71 29  0
 1 1 0 4183836 11172  447   0   0   0 444   0 427   0   0  667 3198 1128 93  7  0
 1 1 0 4183836 11172  411   0   0   0 408   0 459   0   0  699 3424 1039 89 11  0
 1 1 0 4183836 11172  428   0   0   0 425   0 469   0   0  713 3430 1039 96  4  0
 2 0 0 4183832 2 3544   0   0   0 3285   0 296   0   0  535 6820 990 67 32  1
 3 0 0 4184036 11084 10155   0   0   0 9427   0  60   0   0  298 14837 1026 45 55  0
 2 0 0 4184044 11092 7856   0   0   0 7292   0   0   0   0  242 12077 1013 52 48  0
 3 0 0 4183588 11216 13700   0   0   0 12718   0   0   0   0  239 18826 960 26 74  0
 3 0 0 4184056 6 12824   0   0   0 11880   0   5   0   0  245 17635 977 35 65  0
 2 0 0 4183836 11164 8604   0   0   0 8011   0   1   0   0  242 12450 1008 55 45  0
 2 1 0 4183900 11168 5770   0   0   0 5386   0 189   0   0  428 9073 1241 57 43  0
 1 1 0 4183836 11168  497   0   0   0 489   0 472   0   0  712 3006 1107 91  9  0
 1 2 0 4183496 11136  502   0   0   0 483   0 577   0   0  829 3449 1059 88 12  0
 1 1 0 4183496 11156 1444   0   0   0 1365   0 538   0   0  779 4620 1049 86 14  0
 3 0 0 4183248 11120 11937   0   0   0 11068   0 107   0   0  344 16580 892 22 78  0
 2 0 0 4183308 11132 15356   0   0   0 14213   0   1   0   0  241 20742 979 18 82  0
 3 0 0 4183704 11048 12665   0   0   0 11713   0   0   0   0  239 17461 990 28 72  0
 2 0 0 4183492 11092 10878   0   0   0 10126   0  21   0   0  259 15238 986 45 55  0
 2 0 0 4183496 2 12151   0   0   0 11276   0  50   0   0  289 16767 982 28 72  0
 2 1 0 4183496 11144  422   0   0   0 425   0 510   0   0  756 2444 1217 84 16  0
 1 1 0 4183496 11100  475   0   0   0 462   0 575   0   0  820 3299 1131 87 13  0
 1 1 0 4183496 11092 1024   0   0   0 976   0 550   0   0  792 4130 1049 88 12  0
 3 0 0 4183648 11104 7254   0   0   0 6747   0 291   0   0  530 11196 949 53 47  0
 2 1 0 4183768 11140 12570   0   0   0 11678   0   5   0   0  249 17381 1002 33 67  0
 2 0 0 4183832 11028 13877   0   0   0 12846   0   7   0   0  249 18952 986 25 75  0
 2 0 0 4183648 11028 12263   0   0   0 11400   0  16   0   0  259 16964 1015 38 62  0
 3 0 0 4183768 11036 11108   0   0   0 10353   0   5   0   0  244 15567 1017 32 68  0
 3 0 0 4183836 10952 5932   0   0   0 5539   0 190   0   0  429 9087 1153 55 45  0
 2 1 0 4183836 10900  475   0   0   0 459   0 542   0   0  782 2865 1257 91  9  0
 1 2 0 4183768 10936  766   0   0   0 749   0 562   0   0  809 3732 1219 87 13  0
 1 1 0 4184312 10900  853   0   0   0 803   0 557   0   0  799 4189 1142 90 10  0
 2 0 0 4184124 10812 11847   0   0   0 10971   0 101   0   0  340 16642 969 29 71  0
 2 0 0 4184520 10720 8599   0   0   0 7976   0   6   0   0  244 13098 1065 48 52  0
 2 0 0 4184312 10732 7968   0   0   0 7410   0   5   0   0  242 12228 1034 50 49  1


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: {a}sync updates (was Re: make install trick)

[Dmitrij Tejblum]

"David Schwartz" wrote:
>   We're talking about the special case of small root partitions, such that
> softupdates inability to make empty space available quickly can make the
> difference between a major operation's success or failure.
> 
>   This is almost impossible on a 1.8Gb root partition.

[sorry, cannot resist...]


Once upon a time, a month or so ago, there were ~30G of free space on
our 130G filesystem (with softupdates). An important application that 
was going to create a ~35G file was running. It already written out 1G 
or so. My colleague called me and sayed: "I removed a 10G of files TWO 
HOURS ago and the space didn't free up yet!!! The free space isn't 
going to appear!!! What do we do now???"

Well, after I stopped the application with kill -STOP, and temporary 
killed off another I/O consuming program, the free space started to 
appear and after several minutes there were >40G of free space.


I think that the problem in this particular case was inability of the 
syncer to run as fast as it supposed to. It is assummed that syncer 
fsync 1/30 of all files and process the softupdates worklist every 
second. If there are several I/O bound processes running, the syncer 
will not have enough I/O bandwidth to do this job in the required speed. 
Perhaps running several syncer processes could help. (OTOH, the machine 
in question is running a quite old version of FreeBSD-CURRENT, it is 
possible things are already better. I don't see serious changes in 
softupdates code from that moment, tough. It is also possible that the 
machine is mistuned in some way, but I don't know how :-()

Dima




To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: {a}sync updates (was Re: make install trick)

[Matthew Dillon]

:>  We're talking about the special case of small root partitions, such that
:> softupdates inability to make empty space available quickly can make the
:> difference between a major operation's success or failure.
:> 
:>  This is almost impossible on a 1.8Gb root partition. 
:
:Again why?  What's the difference between a small / and a 1.8GB (byte not
:...
:Why would I be so concerned?  If I don't expect to need that 15M then,
:I've sized my partition just right.  Don't put cares in my basket that 
:...
:-- 
:-- David([EMAIL PROTECTED])

I think the argument has become somewhat skewed.  The softupdates bug
occurs when a filesystem fills up, it doesn't really matter how small or
large the filesystem is.

What matters more is how often a partition is actually written to and
how likely the chance of the partition filling up.

Personally speaking, I tend to use small (64-128MB) root partitions
with a separately mounted /usr and /var (and /tmp a softlink to /var/tmp).
In fact, I usually separate out /var/tmp as well.  I do this simply to
reduce the amount of writing that occurs on the root partition in order
to ensure that I don't lose it accidently.

This has saved my butt on innumerable occassions... there is something
to be said for being able to boot into a workable shell when you've
blown up the rest of the system!  With my configuration I feel perfectly
safe enabling softupdates on root.  In other configurations, such as 
having /usr and /var on the same partition as root, I might not
feel as safe.

-Matt
Matthew Dillon 
<[EMAIL PROTECTED]>


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: {a}sync updates (was Re: make install trick)

[David O'Brien]

>   We're talking about the special case of small root partitions, such that
> softupdates inability to make empty space available quickly can make the
> difference between a major operation's success or failure.
> 
>   This is almost impossible on a 1.8Gb root partition. 

Again why?  What's the difference between a small / and a 1.8GB (byte not
bit) one?  There is nothing here to back this up.  Not enough space is
not enough space reguardless how much you started out with.


> And if you had a 1.8Gb partition with only 15Mb free, the last thing
> you'd care about is how softupdates will handle the situation.

Why would I be so concerned?  If I don't expect to need that 15M then,
I've sized my partition just right.  Don't put cares in my basket that 
don't need to be there.  More uninstatiated opions.
 
-- 
-- David([EMAIL PROTECTED])


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

RE: {a}sync updates (was Re: make install trick)

[David Schwartz]

> On Thu, Oct 07, 1999 at 03:15:03PM -0700, David Schwartz wrote:
> > > > There should be fairly few writes to the root partition, so having
> > > An opionion.  I use the HP workstation model where my / is
> 1800M.  I have
>
> > You are not disagreeing with him, David. You are just talking about
> > another scenario other than the one under discussion.
> > He was talking about the case where root is small. This
> whole discussion
> >   was about how softupdates behaves in the subcase of small
> root partitions.
>
> This discussion was NOT about "how softupdates behaves in the subcase of
> small root partitions"  Imp was having problems in the face of
> softupdates on a full /.  The problem exists *reguardless* of how big /
> is, the issues is % free.

No, this is a different issue. The problem was not that the filesystem was
full, but that the fill rate was comparable to the amount of 'empty' space.

> > If you have a 1.8Gb root partition that also includes /var
> and /usr, this
> > whole discussion is irrelevant.
>
> Why??  Because / can now not fill up?  I've installed enough
> KDE/GNOME/teTeX/etc... ports (plus /usr/{ports,src}) that I have acutally
> filled it up before.

Yes, but if you fill up your root partition, there's nothing that can be
done. Full is full.

We're talking about the special case of small root partitions, such that
softupdates inability to make empty space available quickly can make the
difference between a major operation's success or failure.

This is almost impossible on a 1.8Gb root partition. And if you had a 1.8Gb
partition with only 15Mb free, the last thing you'd care about is how
softupdates will handle the situation.

DS



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: {a}sync updates (was Re: make install trick)

[David O'Brien]

On Thu, Oct 07, 1999 at 03:15:03PM -0700, David Schwartz wrote:
> > > There should be fairly few writes to the root partition, so having
> > An opionion.  I use the HP workstation model where my / is 1800M.  I have

>   You are not disagreeing with him, David. You are just talking about
>   another scenario other than the one under discussion.
>   He was talking about the case where root is small. This whole discussion
>   was about how softupdates behaves in the subcase of small root partitions.

This discussion was NOT about "how softupdates behaves in the subcase of
small root partitions"  Imp was having problems in the face of
softupdates on a full /.  The problem exists *reguardless* of how big /
is, the issues is % free.

>   If you have a 1.8Gb root partition that also includes /var and /usr, this
> whole discussion is irrelevant.

Why??  Because / can now not fill up?  I've installed enough
KDE/GNOME/teTeX/etc... ports (plus /usr/{ports,src}) that I have acutally
filled it up before.  

(And before I'm attacked for my organization -- if I can get it back from
the CDROM and it lives in /usr, it is on the / file systems which I don't
back up, as there is no use to.)

-- 
-- David([EMAIL PROTECTED])


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[David O'Brien]

>If you've done your job right, it can be mounted read-only.  This
> makes it harder for someone to break into the machine and obtain root
> access, because now they have to be root to unmount /usr and remount
> it read-write, so that they can put their trojan script on there that
> they're hoping you'll execute.

AND just how are crackers going to write their trojan's in my root owned
/usr (and remember root now owns the binaries in /usr) w/o *already*
being root.  This is just as weak as the argument that BPF makes a box
more vulnerable to having a rouge sniffer running on it.


>   You're right that this is a somewhat religious issue, however, if 
> you're going to run a huge root filesystem, then you are more likely 
> to get what you deserve if /usr or one of the other directories on 
> the root filesystem get trashed or fill up.

And just what do I "deserve"?  Fuh!  Yea, as some said, lets go with a
30MB / so you can't even have room for a second kernel.  You should see
how fscked up Beast.freebsd.org is because of all the /, /usr, /var,
/tmp, etc, were mis-sized.  If I "deserve" something, then what's the
proper sizes for these?  I can tell you I run out of space on / a lot
less my way and have space where I need it, than I do on machines with
the millions of partitions.

Fuh!

-- 
-- David([EMAIL PROTECTED])


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Brad Knowles]

At 3:21 PM -0700 1999/10/7, David O'Brien wrote:

>   HP and SGI workstations have a single huge /.  Why do you
> need /usr seperate from / when you aren't diskless (or /usr'less)?

If you've done your job right, it can be mounted read-only.  This 
makes it harder for someone to break into the machine and obtain root 
access, because now they have to be root to unmount /usr and remount 
it read-write, so that they can put their trojan script on there that 
they're hoping you'll execute.

I've admin'ed my share of HP and SGI machines in the past, and 
I've never used the standard partition configuration, just like I 
don't use the standard partition configuration for Solaris.  IMO, 
none of them are right, and they're wrong for the wrong reasons.


You're right that this is a somewhat religious issue, however, if 
you're going to run a huge root filesystem, then you are more likely 
to get what you deserve if /usr or one of the other directories on 
the root filesystem get trashed or fill up.

-- 
   These are my opinions -- not to be taken as official Skynet policy
  
|o| Brad Knowles, <[EMAIL PROTECTED]>Belgacom Skynet NV/SA |o|
|o| Systems Architect, News & FTP Admin  Rue Col. Bourg, 124   |o|
|o| Phone/Fax: +32-2-706.11.11/12.49 B-1140 Brussels   |o|
|o| http://www.skynet.be Belgium   |o|
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
  Unix is like a wigwam -- no Gates, no Windows, and an Apache inside.
   Unix is very user-friendly.  It's just picky who its friends are.


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: {a}sync updates (was Re: make install trick)

[Peter Jeremy]

On 1999-Oct-08 08:13:12 +1000, David O'Brien wrote:
>> >mount(8):
>> >  syncAll I/O to the file system should be done synchronously.
>> 
>> How detailed should the man page be?  If it stated "all file data will
>> be written synchronously, but inodes where the only update is atime
>> and free block bitmaps are written asynchronously", would that be any
>> clearer to a user who didn't have a detailed understanding of UFS?
>
>Yes.  I know the difference between sync/async and data/metadata.

My point was that the average user probably doesn't.  I agree that
the current description glosses over the difference (and probably
shouldn't).  We need to strike a balance between providing enough
detail for the knowledgeable user (who wants to know exactly what
a sync mount does to different types of writes within the FS) and
the novice user (who doesn't understand the details of UFS and
is more likely to become confused).

IMHO, `sync' does behave in a reasonable manner.  I'm not sure how I'd
go about explaining its behaviour to someone who didn't understand the
UFS though.

>  Guess I go off editing mount(8) again.

I was going to suggest that the relationship between the sync option
in mount(2,8) and O_FSYNC in open(2) be noted.  Only problem is that
O_FSYNC isn't documented :-(.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[David O'Brien]

>   It was my understanding that it was standard recommended practice 
> practice pretty much across the board to create the following 
> separate filesystems:
> 
>   /
>   /tmp (perhaps an mfs, perhaps softupdates, or whatever)
>   /usr
>   /var
>   /var/tmp
>   /home (or wherever you're going to store user directories)
>   And that most people also then created a separate filesystem for 
> /usr/local or /opt, or wherever they're going to store the additional 

You are entering religion.  I despise, HIGHLY DESPISE, all the
partitions.  I don't care for the fragmentation and PITA when upgrading
it leads to.  HP and SGI workstations have a single huge /.  Why do you
need /usr seperate from / when you aren't diskless (or /usr'less)?  Look
at the historic reasons for this division and see if it still makes sense
to you today.  (and before someone misreads this, yes, my /home is a
seperate partition and my /tmp is MFS)

-- 
-- David([EMAIL PROTECTED])


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

RE: {a}sync updates (was Re: make install trick)

[David Schwartz]

> > There should be fairly few writes to the root partition, so having
>
> An opionion.  I use the HP workstation model where my / is 1800M.  I have
> no use for /var and /usr and find them simply stupid in today's world.
> (except for ISP's where there is cause for a septerate /var).
>
> Lets stick to facts.
>
> --
> -- David([EMAIL PROTECTED])

You are not disagreeing with him, David. You are just talking about another
scenario other than the one under discussion.

He was talking about the case where root is small. This whole discussion
was about how softupdates behaves in the subcase of small root partitions.

If you have a 1.8Gb root partition that also includes /var and /usr, this
whole discussion is irrelevant.

DS



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: {a}sync updates (was Re: make install trick)

[David O'Brien]

> >mount(8):
> >  syncAll I/O to the file system should be done synchronously.
> 
> How detailed should the man page be?  If it stated "all file data will
> be written synchronously, but inodes where the only update is atime
> and free block bitmaps are written asynchronously", would that be any
> clearer to a user who didn't have a detailed understanding of UFS?

Yes.  I know the difference between sync/async and data/metadata.  I
haven't however, done a though wall-thru of the UFS code.  If the manpage
says "All I/O", then the system should do *ALL* I/O sync.  This isn't
Linux.  Guess I go off editing mount(8) again.

-- 
-- David([EMAIL PROTECTED])


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: {a}sync updates (was Re: make install trick)

[David O'Brien]

> There should be fairly few writes to the root partition, so having

An opionion.  I use the HP workstation model where my / is 1800M.  I have
no use for /var and /usr and find them simply stupid in today's world.
(except for ISP's where there is cause for a septerate /var).

Lets stick to facts.

-- 
-- David([EMAIL PROTECTED])


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: {a}sync updates (was Re: make install trick)

[Matthew Thyer]

Maybe the best solution is the following:

- leave "sync" with its current behaviour
- create a sysctl to make it truely synchronous (I was thinking of a new
mount option but thats overkill) and have the documentation for that
sysctl state the performance hit and recommend that the filesystem be
mounted with "noatime" when this sysctl is on.

The sysctl could have three levels:

- off
- on for atime updates
- on for atime updates and free block bitmap updates

On Thu, 7 Oct 1999, Peter Jeremy wrote:

> On 1999-Oct-07 09:15:42 +1000, Matthew D. Fuller wrote:
> >Is this good, bad, ugly, or just inconsistent?  On the one hand, you can
> >argue that 'sync should be sync should be sync, I don't bloody care, just
> >don't do anything async at all', since that's what it's supposed to do:
> >mount(8):
> >  syncAll I/O to the file system should be done synchronously.
> 
> How detailed should the man page be?  If it stated "all file data will
> be written synchronously, but inodes where the only update is atime
> and free block bitmaps are written asynchronously", would that be any
> clearer to a user who didn't have a detailed understanding of UFS?
> If you would like it to say something different, write some patches
> and send them in as a PR (keeping in mind phk's recent e-mail about
> green bikesheds).
> 
> >  sync atime updates will slow it
> >down, but on the flip side, if you're mounting sync in the first place
> >you don't care much for speed anyway.
> 
> There should be fairly few writes to the root partition, so having
> these writes synchronous is not a big performance hit.  On the other
> hand, there are probably a _lot_ of read accesses to devices in /dev
> and files in /bin (how many of your scripts begin #!/bin/sh?).  Unless
> you specify NOATIME, each of these read accesses implies an atime
> update within the inode.  Making these synchronous probably would
> be a big performance hit.
> 
> Peter
> 

-- 
/===\
| Work: [EMAIL PROTECTED] | Home: [EMAIL PROTECTED] |
\===/
"If it is true that our Universe has a zero net value for all conserved
quantities, then it may simply be a fluctuation of the vacuum of some
larger space in which our Universe is imbedded. In answer to the
question of why it happened, I offer the modest proposal that our
Universe is simply one of those things which happen from time to time."
 E. P. Tryon   from "Nature" Vol.246 Dec.14, 1973



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: {a}sync updates (was Re: make install trick)

[Matthew D. Fuller]

On Thu, Oct 07, 1999 at 11:57:26AM +1000, a little birdie told me
that Peter Jeremy remarked
> 
> How detailed should the man page be?

Exactly my query in writing this  ;>


> If it stated "all file data will
> be written synchronously, but inodes where the only update is atime
> and free block bitmaps are written asynchronously", would that be any
> clearer to a user who didn't have a detailed understanding of UFS?
> If you would like it to say something different, write some patches
> and send them in as a PR (keeping in mind phk's recent e-mail about
> green bikesheds).

I'm still stewing on what should be with it; what we have works fine, if
being slightly inconsistent in an obscure way.  I'm trolling for ideas on
whether well enough should be left alone (since there's obviously an
incredibly small percentage of people USING sync as a mountop), or
whether a footnote should be added somewhere (I lean toward mount(2)
instead of mount(8) myself, with a possible xref in mount(8)).

I'll see what I think of, and possibly have some diffs tomorrow.


> There should be fairly few writes to the root partition, so having
> these writes synchronous is not a big performance hit.  On the other
> hand, there are probably a _lot_ of read accesses to devices in /dev
> and files in /bin (how many of your scripts begin #!/bin/sh?).  Unless
> you specify NOATIME, each of these read accesses implies an atime
> update within the inode.  Making these synchronous probably would
> be a big performance hit.

This is why I haven't screamed for them to be sync-tified...




-- 
Matthew Fuller (MF4839) |[EMAIL PROTECTED]
Unix Systems Administrator  |[EMAIL PROTECTED]
Specializing in FreeBSD |http://www.over-yonder.net/
FutureSouth Communications  |ISPHelp ISP Consulting

"The only reason I'm burning my candle at both ends, is because I
  haven't figured out how to light the middle yet"


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: {a}sync updates (was Re: make install trick)

[Peter Jeremy]

On 1999-Oct-07 09:15:42 +1000, Matthew D. Fuller wrote:
>Is this good, bad, ugly, or just inconsistent?  On the one hand, you can
>argue that 'sync should be sync should be sync, I don't bloody care, just
>don't do anything async at all', since that's what it's supposed to do:
>mount(8):
>  syncAll I/O to the file system should be done synchronously.

How detailed should the man page be?  If it stated "all file data will
be written synchronously, but inodes where the only update is atime
and free block bitmaps are written asynchronously", would that be any
clearer to a user who didn't have a detailed understanding of UFS?
If you would like it to say something different, write some patches
and send them in as a PR (keeping in mind phk's recent e-mail about
green bikesheds).

>  sync atime updates will slow it
>down, but on the flip side, if you're mounting sync in the first place
>you don't care much for speed anyway.

There should be fairly few writes to the root partition, so having
these writes synchronous is not a big performance hit.  On the other
hand, there are probably a _lot_ of read accesses to devices in /dev
and files in /bin (how many of your scripts begin #!/bin/sh?).  Unless
you specify NOATIME, each of these read accesses implies an atime
update within the inode.  Making these synchronous probably would
be a big performance hit.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: {a}sync updates (was Re: make install trick)

[Matthew Dillon]

:mount(8):
: syncAll I/O to the file system should be done synchronously.
:
:On the gripping hand, you can say, 'this is an ATIME update, there's no
:way its presence or lack thereof can do anything bad to the filesystem,
:so let it be async since it takes extra work to make it sync'.
:
:Does anyone have any feeling either way on this?  I, unfortunately, seem
:to have strong feelings BOTH ways...  sync atime updates will slow it
:down, but on the flip side, if you're mounting sync in the first place
:you don't care much for speed anyway.
:
:Thoughts?
:Matthew Fuller (MF4839) |[EMAIL PROTECTED]

Well, you don't gain anything by making atime updates sync, and you
lose a lot, so why do it?  If you are worried about protecting the root
drive from a crash, mount it read-only (and put /dev on an MFS mount)
or mount it noatime.
 
-Matt
Matthew Dillon 
<[EMAIL PROTECTED]>


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Bruce Evans]

> /dev/da0s1a on / (local, synchronous, writes: sync 32 async 15100)
>  ^^^
> 
> Though I'm still waiting for an explanation of WHY exactly I have async
> writes on a sync partition.   Nobody yet has said anything but 'that's
> interesting...'.  A direction to look would be helpful.

The sync flag only affects data writes and block allocation for data
writes.  Metadata is normally always written synchronously, except for
safe optimisations.  Safe optimisations include never writing inode or
block bitmaps (fsck can recover these), not writing some indirect blocks
(correct block numbers are not very useful until the blocks have been
written), and not writing inodes in many cases where only timestamps
have changed.

You seem to have a lot of async writes.  This may be caused by using
soft updates.  Soft updates causes many (or most or all) metadata
updates to be done using delayed writes.  Using the sync flag with
soft updates doesn't make much sense and may be harmfull (there doesn't
seem to be anything to prevent data writes being done long before the
metadata points to the data).

Smaller number of async writes may be caused by metadata write
optimisations.

Bruce



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

{a}sync updates (was Re: make install trick)

[Matthew D. Fuller]

Thank you, this is *EXACTLY* what I was looking for   :)

On Thu, Oct 07, 1999 at 08:59:00AM +1000, a little birdie told me
that Peter Jeremy remarked
> 
> As far as I can tell, the net effect is that inode access time updates
> will remain async writes into the filesystem.
> 
> An easy way to tell would be to use NOATIME and see if you're still
> getting async writes.  (Or any writes at all).

This does appear to be it.
Testing such things as (cat /kernel >> /dev/null && sync && sync)
preceded and followed by a 'mount' call in another window does show the
'async' counter incrementing, and when I remount it noatime it no longer
does.

Is this good, bad, ugly, or just inconsistent?  On the one hand, you can
argue that 'sync should be sync should be sync, I don't bloody care, just
don't do anything async at all', since that's what it's supposed to do:
mount(8):
 syncAll I/O to the file system should be done synchronously.

On the gripping hand, you can say, 'this is an ATIME update, there's no
way its presence or lack thereof can do anything bad to the filesystem,
so let it be async since it takes extra work to make it sync'.


Does anyone have any feeling either way on this?  I, unfortunately, seem
to have strong feelings BOTH ways...  sync atime updates will slow it
down, but on the flip side, if you're mounting sync in the first place
you don't care much for speed anyway.

Thoughts?




-- 
Matthew Fuller (MF4839) |[EMAIL PROTECTED]
Unix Systems Administrator  |[EMAIL PROTECTED]
Specializing in FreeBSD |http://www.over-yonder.net/
FutureSouth Communications  |ISPHelp ISP Consulting

"The only reason I'm burning my candle at both ends, is because I
  haven't figured out how to light the middle yet"


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Ben Rosengart]

On Wed, 6 Oct 1999, Matthew D. Fuller wrote:

> Indeed.
> Thus:
> /dev/da0s1a on / (local, synchronous, writes: sync 32 async 15100)
>  ^^^
> 
> Though I'm still waiting for an explanation of WHY exactly I have async
> writes on a sync partition.   Nobody yet has said anything but 'that's
> interesting...'.  A direction to look would be helpful.

I believe "synchronous" refers only to metadata writes.  So it would
follow that the async writes are non-metadata writes.

--
 Ben Rosengart

UNIX Systems Engineer, Skunk Group
StarMedia Network, Inc.



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Peter Jeremy]

On 1999-Oct-07 06:44:19 +1000, Matthew D. Fuller wrote:
>/dev/da0s1a on / (local, synchronous, writes: sync 32 async 15100)
> ^^^
>
>Though I'm still waiting for an explanation of WHY exactly I have async
>writes on a sync partition.   Nobody yet has said anything but 'that's
>interesting...'.  A direction to look would be helpful.

You get a synchronous mount by passing MNT_SYNCHRONOUS to mount(2).
Within the kernel, MNT_SYNCHRONOUS is used (basically only) within
kern/vfs_vnops.c:vn_write().  There it adds IO_SYNC to the write
request (the same as if the file has O_FSYNC enabled).  IO_SYNC will
be ignored if MNT_ASYNC is specified (which it isn't here).  The
write request goes through ufs/ufs/ufs_readwrite.c:WRITE(), which
passes it to bwrite() if it's IO_SYNC.

Inode access time updates (controlled by MNT_NOATIME, which becomes
~IN_ACCESS, which becomes IN_MODIFIED) don't go through vn_write() and
wind up in bdwrite().

As far as I can tell, the net effect is that inode access time updates
will remain async writes into the filesystem.

An easy way to tell would be to use NOATIME and see if you're still
getting async writes.  (Or any writes at all).

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: pfft Re: make install trick

[Peter Jeremy]

On 1999-Oct-07 01:51:52 +1000, Alfred Perlstein wrote:
>First post:
>  hey, anyone having crashes when doing installworld might like
>  my INSTALL="sync && install -C" trick

I haven't kept that post.  My recollection is that the problem was
installworld dying, not the system crashing.  If I misread or
mis-remember the post, my apologies.

>no matter which option you choose you can still get bitten by
>softupdates when you do run out of space even by a trusted
>user such as root.

Softupdates reporting ENOSPC soon after a delete is a known bug.
At this stage the recommended workaround (according to Kirk) is
not to run softupdates on filesystems where this condition is
likely.  If anyone wants to fix it, patches would be welcomed
by Kirk or Julian.

If softupdates is causing panics, that is an extremely undesirable
bug.  I haven't been bitten (yet) and don't have a solution.

My view is that the remainder of the thread has been discussing the
relative merits of different ways to avoid the problem.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Peter Jeremy]

On 1999-Oct-06 16:14:19 +1000, Brian Somers wrote:
>> On 1999-Oct-06 09:55:26 +1000, Alfred Perlstein wrote:
>> >I've seen softupdates nearly eliminate disk io for systems that used
>> >an abmornal amount of temp files, but the fact that it can destabilize
>> >a system worries me greatly.
>> 
>> What do you mean by `destabilize'?  There are bugs in softupdates
>> which mean that an application may receive ENOSPC when writing to a
>> filesystem even though space on that filesystem has been recently
>> freed.  Any application that cannot handle this situation is _broken_.
>[.]
>
>:-]  You're joking right ?  Most applications don't even *notice* 
>the situation let alone handle it.

I wasn't really joking.  Applications _should_ handle this situation.
Most coding standards tell you to check for error conditions.  Most
programmers (including myself most of the time) just don't bother.
(One reason being that C suffers from extreme code bloat and
substantial loss of readability.  C++ exceptions are a definite
improvement here).

>  Whaddaya do when /var/log runs 
>out of space ?  Show me an app that handles it.

The only app that normally writes in /var/log is syslogd.  And the
behaviour you want in this case depends on the sysadmin's level of
paranoia (anything from `ignore it' to `halt the system').

>I guess you can argue the case, but in real life, running out of any 
>machine resource can cause all sorts of possibly unrecoverable 
>problems.
Agreed.  One reason I (and presumably others) don't bother to check
for some errors is that it's not clear what you can or should do if
you get the error.

>  IMHO the best way to deal with it in a generic way is to 
>have the give-me-the-resource function block if it really thinks it's 
>a temporary thing.

How do you work out whether the resource shortage is temporary or not?
This situation is orthogonal to the ever-popular `out of swapspace'
issue (normally brought up in conjunction with swap overcommit).  The
difficulty with any recovery strategy is avoiding deadlocks.

(I suspect that a major reason why root is not constrained by UFS
MINFREE was to try and avoid the situation where the _system_ (as
against the users) ran out of disk space).

>In the case of softupdtes, I'd be surprised if it couldn't easily 
>figure out that the problem is *probably* transient and block,

To quote an excerpt from the response I got from Kirk when I
asked him about this problem:

: I experimented with keeping a count of space that was pending
:removal. If the filesystem was about to return a space full message,
:it would check the pending count for the filesystem and if there was
:space that was going to show up in the future, it would request that
:the space freeing be expedited then go to sleep and wait for it. The
:problem is that the space freeing can take up to a minute (typically
:more like 30 seconds) during which time the file's inode is
:locked. If the locked file is say a log file, then you can get a
:temporary lock race to the root which make for really terrible
:perceived performance. If the inode is unlocked during the wait, then
:file inconsistencies can sneak in. So, the short answer is that you
:should not run soft updates on filesystems where you are running with
:less than a 60 second reserve of free space.

Note the last sentence...

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Matthew Dillon]

:Indeed.
:Thus:
:/dev/da0s1a on / (local, synchronous, writes: sync 32 async 15100)
: ^^^
:
:Though I'm still waiting for an explanation of WHY exactly I have async
:writes on a sync partition.   Nobody yet has said anything but 'that's
:interesting...'.  A direction to look would be helpful.
:...
:Matthew Fuller (MF4839) |[EMAIL PROTECTED]
:

Access time updates perhaps?  Try mounting with the 'noatime' option.

Also, note that softupdates is *safer* then a synchronous mount in
regards to ensuring filesystem consistancy.

-Matt
Matthew Dillon 
<[EMAIL PROTECTED]>


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Matthew D. Fuller]

On Wed, Oct 06, 1999 at 04:18:15PM -0500, a little birdie told me
that Kevin Day remarked
> >
> > /dev/da0s1a on / (local, synchronous, writes: sync 32 async 15100)
> 
> My understanding was that that was just a indication of writes that were
> able to be done asynchronously without any risk, so they were done async.
> 
> (sync isn't purely sync, only synchronous when it's required for integrity)

I was given to understand that while the default mountop follows these
conventions, explicit 'sync' meant SYNC meant SYNC meant SYNC.

This is my root filesystem.  It gets written to when I edit a file under
/etc or do an installworld.  I don't *CARE* how slow it is, I want to
know that it's solid, consistent, and complete.  Just 'consistent' isn't
enough.  No write is able to be done async without risk when I'm
explicitly saying 'write everything synchronously'.




-- 
Matthew Fuller (MF4839) |[EMAIL PROTECTED]
Unix Systems Administrator  |[EMAIL PROTECTED]
Specializing in FreeBSD |http://www.over-yonder.net/
FutureSouth Communications  |ISPHelp ISP Consulting

"The only reason I'm burning my candle at both ends, is because I
  haven't figured out how to light the middle yet"


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Kevin Day]

> 
> On Wed, Oct 06, 1999 at 02:57:23PM +1000, a little birdie told me
> that Peter Jeremy remarked
> > 
> > I guess we disagree on this.  My feeling is that write activity on
> > root should be minimised to minimise the risk that root will be
> > inconsistent following a crash.
> 
> Indeed.
> Thus:
> /dev/da0s1a on / (local, synchronous, writes: sync 32 async 15100)
>  ^^^
> 
> Though I'm still waiting for an explanation of WHY exactly I have async
> writes on a sync partition.   Nobody yet has said anything but 'that's
> interesting...'.  A direction to look would be helpful.
> 

My understanding was that that was just a indication of writes that were
able to be done asynchronously without any risk, so they were done async.

(sync isn't purely sync, only synchronous when it's required for integrity)

Kevin


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Matthew D. Fuller]

On Wed, Oct 06, 1999 at 02:57:23PM +1000, a little birdie told me
that Peter Jeremy remarked
> 
> I guess we disagree on this.  My feeling is that write activity on
> root should be minimised to minimise the risk that root will be
> inconsistent following a crash.

Indeed.
Thus:
/dev/da0s1a on / (local, synchronous, writes: sync 32 async 15100)
 ^^^

Though I'm still waiting for an explanation of WHY exactly I have async
writes on a sync partition.   Nobody yet has said anything but 'that's
interesting...'.  A direction to look would be helpful.




-- 
Matthew Fuller (MF4839) |[EMAIL PROTECTED]
Unix Systems Administrator  |[EMAIL PROTECTED]
Specializing in FreeBSD |http://www.over-yonder.net/
FutureSouth Communications  |ISPHelp ISP Consulting

"The only reason I'm burning my candle at both ends, is because I
  haven't figured out how to light the middle yet"


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Brad Knowles]

At 12:22 PM +0200 1999/10/6, Brad Knowles wrote:

>   Is there something fundamental I'm missing here?  I thought
> that this sort of thing was taught in SysAdmin 101

My sincerest (and public) apologies to Alfred.

There is something pretty fundamental that I was missing -- this 
problem is not restricted to just a softupdates filesystem filling 
up, and if it's the root filesystem then there can be a crash.  No, 
this problem appears to be that if *any* softupdates filesystem fills 
up, it can crash the system.


Now that I understand what it was I was missing, I'm much more 
reluctant to use softupdates in general, at least until this problem 
is fixed.


Thanks for understanding, and helping me to more clearly 
understand the problem!

-- 
   These are my opinions -- not to be taken as official Skynet policy
  
|o| Brad Knowles, <[EMAIL PROTECTED]>Belgacom Skynet NV/SA |o|
|o| Systems Architect, News & FTP Admin  Rue Col. Bourg, 124   |o|
|o| Phone/Fax: +32-2-706.11.11/12.49 B-1140 Brussels   |o|
|o| http://www.skynet.be Belgium   |o|
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
  Unix is like a wigwam -- no Gates, no Windows, and an Apache inside.
   Unix is very user-friendly.  It's just picky who its friends are.


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Ollivier Robert]

According to Nate Williams:
> So, are you suggesting make /tmp it's own disk, otherwise anytime you do
> development alot of writes are done to /.

Of course /tmp should NOT be on /. Either in its separate FS or on another
partition through a symlink.

/ should be as small and as write-free as possible. Mine is too small now for
a 4.0 system (/ has grown a bit since 2.0... :)) though.

> And, if you do lots of development, then you'll have the same problem on
> /tmp as you did on / unless you waste a huge disk for /tmp. :(

It doesn't have to be that huge and -pipe helps a lot for C/C++ :)
-- 
Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- [EMAIL PROTECTED]
FreeBSD keltia.freenix.fr 4.0-CURRENT #74: Thu Sep  9 00:20:51 CEST 1999



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Garrett Wollman]

< said:

>   Forgive me if I'm misunderstanding something here, but isn't 
> having /tmp on the root filesystem just inviting a denial-of-service 
> attack on yourself?

Only if you let random lusers log in to your machine.

-GAWollman

--
Garrett A. Wollman   | O Siem / We are all family / O Siem / We're all the same
[EMAIL PROTECTED]  | O Siem / The fires of freedom 
Opinions not those of| Dance in the burning flame
MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Brad Knowles]

At 10:51 AM -0400 on 1999/10/6, Garrett Wollman 
<[EMAIL PROTECTED]>  wrote:

> Only if you let random lusers log in to your machine.

That's not the way I interpret some of the previous comments on 
this thread.

It seems to me that not having /tmp on a separate filesystem has, 
indeed, created quite a nice little self-inflicted denial-of-service 
attack on certain people.  ;-)

-- 
   These are my opinions -- not to be taken as official Skynet policy
  
|o| Brad Knowles, <[EMAIL PROTECTED]>Belgacom Skynet NV/SA |o|
|o| Systems Architect, News & FTP Admin  Rue Col. Bourg, 124   |o|
|o| Phone/Fax: +32-2-706.11.11/12.49 B-1140 Brussels   |o|
|o| http://www.skynet.be Belgium   |o|
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
  Unix is like a wigwam -- no Gates, no Windows, and an Apache inside.
   Unix is very user-friendly.  It's just picky who its friends are.


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Brad Knowles]

At 6:33 PM -0700 1999/10/5, Alfred Perlstein wrote:

> Which isn't an option unless you dedicate a partition for /tmp
> which is pretty wasteful imo.

Forgive me if I'm misunderstanding something here, but isn't 
having /tmp on the root filesystem just inviting a denial-of-service 
attack on yourself?  It's bad enough when programs crap out when /tmp 
is full and they can't create the temporary files they demand (vi 
leaps to mind), but when you fill up the root filesystem and the 
whole machine falls over, that seems to be a really bad situation 
that everyone would want to avoid at virtually all costs.

It was my understanding that it was standard recommended practice 
practice pretty much across the board to create the following 
separate filesystems:

/
/tmp (perhaps an mfs, perhaps softupdates, or whatever)
/usr
/var
/var/tmp
/home (or wherever you're going to store user directories)

And that most people also then created a separate filesystem for 
/usr/local or /opt, or wherever they're going to store the additional 
system programs that they're going to be adding.

This also allows you to do nice security things such as mounting 
/tmp, /var, /var/tmp nosuid, etc


Is there something fundamental I'm missing here?  I thought that 
this sort of thing was taught in SysAdmin 101

-- 
   These are my opinions -- not to be taken as official Skynet policy
  
|o| Brad Knowles, <[EMAIL PROTECTED]>Belgacom Skynet NV/SA |o|
|o| Systems Architect, News & FTP Admin  Rue Col. Bourg, 124   |o|
|o| Phone/Fax: +32-2-706.11.11/12.49 B-1140 Brussels   |o|
|o| http://www.skynet.be Belgium   |o|
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
  Unix is like a wigwam -- no Gates, no Windows, and an Apache inside.
   Unix is very user-friendly.  It's just picky who its friends are.


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Brian Somers]

> On 1999-Oct-06 09:55:26 +1000, Alfred Perlstein wrote:
> >I've seen softupdates nearly eliminate disk io for systems that used
> >an abmornal amount of temp files, but the fact that it can destabilize
> >a system worries me greatly.
> 
> What do you mean by `destabilize'?  There are bugs in softupdates
> which mean that an application may receive ENOSPC when writing to a
> filesystem even though space on that filesystem has been recently
> freed.  Any application that cannot handle this situation is _broken_.
[.]

:-]  You're joking right ?  Most applications don't even *notice* 
the situation let alone handle it.  Whaddaya do when /var/log runs 
out of space ?  Show me an app that handles it.

I guess you can argue the case, but in real life, running out of any 
machine resource can cause all sorts of possibly unrecoverable 
problems.  IMHO the best way to deal with it in a generic way is to 
have the give-me-the-resource function block if it really thinks it's 
a temporary thing.

In the case of softupdtes, I'd be surprised if it couldn't easily 
figure out that the problem is *probably* transient and block, but 
of course to figure out exactly how transient it is would probably 
involve about the same amount of effort as just processing the 
softupdates ``todo'' list there and then.

Anyway, I'll shut up now :-I

-- 
Brian <[EMAIL PROTECTED]><[EMAIL PROTECTED]>
     <[EMAIL PROTECTED]>
Don't _EVER_ lose your sense of humour !  <[EMAIL PROTECTED]>




To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Brian Somers]

> > In any case, you should not be doing lots of writes to root, so the
> > lack of softupdates should not be a problem.
> 
> So, are you suggesting make /tmp it's own disk, otherwise anytime you do
> development alot of writes are done to /.
> 
> And, if you do lots of development, then you'll have the same problem on
> /tmp as you did on / unless you waste a huge disk for /tmp. :(

Ah, but if you have /tmp on your root partition, you should really 
have more than 12M of free space:

  $ du -sk /bin /sbin
  3247/bin
  8519/sbin

:-)

Interestingly enough, I believe there's another ``bug'' here in the 
softupdates code.  If softupdates is short on required disk space, it 
sould at least go through the todo list and optimise out what it can, 
possibly reducing the disk space requirements.  If it did this, 
repeating the make install a few times should eventually work as it 
will ultimately end up overwriting things with exactly the same data 
and optimising the whole operation out - but this isn't currently the 
case.

> Nate

-- 
Brian <[EMAIL PROTECTED]><[EMAIL PROTECTED]>
     <[EMAIL PROTECTED]>
Don't _EVER_ lose your sense of humour !  <[EMAIL PROTECTED]>




To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Brian Somers]

[.]
> Please read more of the thread before replying, the fact of
[.]

Now if everyone used mailers that actually wrote correct in-reply-to 
lines, this might be a realistic thing to suggest !

-- 
Brian <[EMAIL PROTECTED]><[EMAIL PROTECTED]>
     <[EMAIL PROTECTED]>
Don't _EVER_ lose your sense of humour !  <[EMAIL PROTECTED]>




To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Peter Jeremy]

On 1999-Oct-06 11:33:51 +1000, Alfred Perlstein wrote:
>
>On Wed, 6 Oct 1999, Peter Jeremy wrote:
>
>> On 1999-Oct-06 09:55:26 +1000, Alfred Perlstein wrote:
>> >I've seen softupdates nearly eliminate disk io for systems that used
>> >an abmornal amount of temp files, but the fact that it can destabilize
>> >a system worries me greatly.
>> 
>> What do you mean by `destabilize'?
...
> softupdates can crash the system when this
>happens, not a mere application, but the entire system can lock up.

I don't recall this coming up in this thread.  Last I recall, that
bug was supposed to be fixed (and Kirk(?) wanted anyone who still
saw it to report it).  Unfortunately, I can't find that thread in
the archives :-(.

>Which isn't an option unless you dedicate a partition for /tmp
>which is pretty wasteful imo.

I guess we disagree on this.  My feeling is that write activity on
root should be minimised to minimise the risk that root will be
inconsistent following a crash.  Secondly, there's a security
viewpoint that users should not have write access anywhere in the root
or /usr filesystems.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Kevin Street]

Nate Williams <[EMAIL PROTECTED]> writes:

> > In any case, you should not be doing lots of writes to root, so the
> > lack of softupdates should not be a problem.
> 
> So, are you suggesting make /tmp it's own disk, otherwise anytime you do
> development alot of writes are done to /.
> 
> And, if you do lots of development, then you'll have the same problem on
> /tmp as you did on / unless you waste a huge disk for /tmp. :(

You could just symlink /tmp to someplace big and softupdateable.  At least
then you can share the free space with another big filesystem.  The only
problem then is that when you boot standalone you either need to be
able to mount wherever you've symlinked it into or rm the symlink and
mkdir /tmp while you're standalone (and remember to put it back before
going multi user).  

In some ways symlinking is better than having /tmp as a mount point.
If it's a real mount point you're likely to create stuff in /tmp
while standalone that can't be seen while you're multi user with a
file system mounted on top of it.  Which leads to "...my / is full and
I can't find the files that are using all the space!"

-- 
Kevin Street
[EMAIL PROTECTED]


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Alfred Perlstein]

On Wed, 6 Oct 1999, Peter Jeremy wrote:

> On 1999-Oct-06 09:55:26 +1000, Alfred Perlstein wrote:
> >I've seen softupdates nearly eliminate disk io for systems that used
> >an abmornal amount of temp files, but the fact that it can destabilize
> >a system worries me greatly.
> 
> What do you mean by `destabilize'?  There are bugs in softupdates
> which mean that an application may receive ENOSPC when writing to a
> filesystem even though space on that filesystem has been recently
> freed.  Any application that cannot handle this situation is _broken_.

Please read more of the thread before replying, the fact of
the matter is that softupdates can crash the system when this
happens, not a mere application, but the entire system can lock up.

> Another option for /tmp - which I forgot last time, and seems to
> avoid the known problems with MFS and softupdates - it to mount
> /tmp async.  Since you're going to blow it all away on the next
> reboot anyway, it doesn't really matter if the a crash trashes the
> FS (which is the major problem with async mounts).

Which isn't an option unless you dedicate a partition for /tmp
which is pretty wasteful imo.

-Alfred



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Peter Jeremy]

On 1999-Oct-06 09:55:26 +1000, Alfred Perlstein wrote:
>I've seen softupdates nearly eliminate disk io for systems that used
>an abmornal amount of temp files, but the fact that it can destabilize
>a system worries me greatly.

What do you mean by `destabilize'?  There are bugs in softupdates
which mean that an application may receive ENOSPC when writing to a
filesystem even though space on that filesystem has been recently
freed.  Any application that cannot handle this situation is _broken_.

Another option for /tmp - which I forgot last time, and seems to
avoid the known problems with MFS and softupdates - it to mount
/tmp async.  Since you're going to blow it all away on the next
reboot anyway, it doesn't really matter if the a crash trashes the
FS (which is the major problem with async mounts).

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Peter Jeremy]

On 1999-Oct-06 09:02:12 +1000, Nate Williams wrote:
>> In any case, you should not be doing lots of writes to root, so the
>> lack of softupdates should not be a problem.
>
>So, are you suggesting make /tmp it's own disk, otherwise anytime you do
>development alot of writes are done to /.

I've pretty well always used a separate /tmp partition.  I've always
used MFS with FreeBSD.  This makes the root partition virtually
read-only (which has robustness and security advantages).

>And, if you do lots of development, then you'll have the same problem on
>/tmp as you did on / unless you waste a huge disk for /tmp. :(

Solutions:
1) Use -pipe
2) Don't use softupdates
3) Use a RAMdisk (eg MFS) instead of a physical disk (which makes the
   previous point irrelevant).

I don't recall seeing anyone mention problems like this (though they
might be on lists I don't read).

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

RE: make install trick

[Alfred Perlstein]

On Tue, 5 Oct 1999, David Schwartz wrote:

> 
> > I have soft updates enabled on a fast machine at work.  make
> > installworld can fill up slash even though it has 15M free before the
> > install.  I think this is a bug in softupdates that it doesn't reclaim
> > space quickly enough or in overflow situations.
> 
>   It's really not a bug, it's just a missing feature. There's no requirement
> that a filesystem reclaim empty space immediately. You really shouldn't be
> using fastupdates on nearly full filesystems -- it doesn't handle that
> situation particularly well.
> 
>   Once could even argue that it's preferable to force the make to abort than
> thrash the filesystem. Though a switch to allow it to thrash might be
> helpful in degenerate cases such as this.
> 
>   Fastupdates is great for the most common case -- a typical /usr or /home
> partition. That's where you care about write performance anyway.

Actually this becomes quite dangerous when used on tmp filesystems, it
used to be that mfs was a good idea for /tmp, but now softupdates
drastically improves performance... however given that a full /tmp
can kill a system that places us in a dilemma now doesn't it?

*shrug*

I've seen softupdates nearly eliminate disk io for systems that used
an abmornal amount of temp files, but the fact that it can destabilize
a system worries me greatly.

Of course I'm trying desperately to understand the softupdates code
right now. :)

-Alfred



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Nate Williams]

> In any case, you should not be doing lots of writes to root, so the
> lack of softupdates should not be a problem.

So, are you suggesting make /tmp it's own disk, otherwise anytime you do
development alot of writes are done to /.

And, if you do lots of development, then you'll have the same problem on
/tmp as you did on / unless you waste a huge disk for /tmp. :(


Nate


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

RE: make install trick

[David Schwartz]

> In message <000101bf0f78$fbe58b40$[EMAIL PROTECTED]> "David
> Schwartz" writes:
> : It's really not a bug, it's just a missing feature. There's
> no requirement
> : that a filesystem reclaim empty space immediately. You really
> shouldn't be
> : using fastupdates on nearly full filesystems -- it doesn't handle that
> : situation particularly well.
>
> Nearly full?  It is a 32M file system with 15M free.  That's not
> nearly full.

Whether that qualifies as 'nearly full' or not depends upon the update
rate.

> The problem is that the update rate is faster than the
> softupdate code can deal with.

I would say a filesystem that adds new files in an amount close to its free
space is nearly full.

> Running sync between each install
> shows that this is a bug.  sync is supposed to be idempotent.

I think anybody reasonable familiar with softupdates understands what
'sync' does in this context. It's not a bug because it is documented
behavior done for a specific reason.

DS



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Warner Losh]

In message <01bf0f86$d8d84670$[EMAIL PROTECTED]> "David Schwartz" writes:
:   I think anybody reasonable familiar with softupdates understands what
: 'sync' does in this context. It's not a bug because it is documented
: behavior done for a specific reason.

It is a bug.  You've just said so.  It isn't a bug that can be fixed
trivially, yet it is still a bug.  now that I know what's going on, I
know why my kludg-eo-round works.  Something that doesn't deal with
reclaiming unused space is broken for that case, no matter how hard it
is to fix.  I also know how to work around it...

Geeze, it isn't like I'm demanding you fix the bug.

Warner


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Peter Jeremy]

On 1999-Oct-06 07:42:39 +1000, Warner Losh wrote:
>Nearly full?  It is a 32M file system with 15M free.  That's not
>nearly full.  The problem is that the update rate is faster than the
>softupdate code can deal with.  Running sync between each install
>shows that this is a bug.

As Ollivier Robert pointed out, it is a known problem.  Kirk says
it's non-trivial to fix (though I'm sure he'll welcome your patches :-).
The suggested work-around is not to run softupdates on root.

In any case, you should not be doing lots of writes to root, so the
lack of softupdates should not be a problem.

Peter
-- 
Peter Jeremy (VK2PJ)[EMAIL PROTECTED]
Alcatel Australia Limited
41 Mandible St  Phone: +61 2 9690 5019
ALEXANDRIA  NSW  2015   Fax:   +61 2 9690 5982


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Warner Losh]

In message <000101bf0f78$fbe58b40$[EMAIL PROTECTED]> "David Schwartz" writes:
:   It's really not a bug, it's just a missing feature. There's no requirement
: that a filesystem reclaim empty space immediately. You really shouldn't be
: using fastupdates on nearly full filesystems -- it doesn't handle that
: situation particularly well.

Nearly full?  It is a 32M file system with 15M free.  That's not
nearly full.  The problem is that the update rate is faster than the
softupdate code can deal with.  Running sync between each install
shows that this is a bug.  sync is supposed to be idempotent.

Warner


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

RE: make install trick

[David Schwartz]

> I have soft updates enabled on a fast machine at work.  make
> installworld can fill up slash even though it has 15M free before the
> install.  I think this is a bug in softupdates that it doesn't reclaim
> space quickly enough or in overflow situations.

It's really not a bug, it's just a missing feature. There's no requirement
that a filesystem reclaim empty space immediately. You really shouldn't be
using fastupdates on nearly full filesystems -- it doesn't handle that
situation particularly well.

Once could even argue that it's preferable to force the make to abort than
thrash the filesystem. Though a switch to allow it to thrash might be
helpful in degenerate cases such as this.

Fastupdates is great for the most common case -- a typical /usr or /home
partition. That's where you care about write performance anyway.

DS



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: make install trick

[Ollivier Robert]

According to Warner Losh:
> install.  I think this is a bug in softupdates that it doesn't reclaim
> space quickly enough or in overflow situations.

Yes, it is a known issue Kirk know of AFAIK. It should probably reclaim the
soon-to-be-freed blocks when it needs them. I've removed softupdates on / for
the moment, it is not that written to on my machines anyway.
-- 
Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- [EMAIL PROTECTED]
FreeBSD keltia.freenix.fr 4.0-CURRENT #74: Thu Sep  9 00:20:51 CEST 1999



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

make install trick

[Warner Losh]

I have soft updates enabled on a fast machine at work.  make
installworld can fill up slash even though it has 15M free before the
install.  I think this is a bug in softupdates that it doesn't reclaim
space quickly enough or in overflow situations.

To work around it, I've used
make INSTALL="/bin/sync && install"
or
make INSTALL="/bin/sync && install -C"

Of course it takes longer this way, but it has worked 4 times in a row
where the last 20 make installworlds failed.

Warner


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message