Re: Install on compact flash

[Stuart Henderson]

On 2015-10-19, Paolo Aglialoro  wrote:
> On Mon, Oct 19, 2015 at 12:27 PM, Stuart Henderson  
> wrote:
>
>> Some devices get chown()ed during normal system operation, see fbtab(5).
>
> Does this mean that write access on files in /dev is limited just to
> permissions change and, therefore, just some bytes of change in the
> filesystem? This seems pretty much acceptable for me in terms of CF
> wear-off, if it does not happen with a high frequence.

Timestamps are updated as well. But I don't think wear-off is really
a problem for any kind of normal use. It's certainly less of a problem
than the need to handle a separate /dev partition and make sure that
things are kept in-sync through OS updates.

If you're worried about it you could get a larger card than you need
and leave some space unused to give a bigger pool for wear-levelling.

Re: Install on compact flash

[Paolo Aglialoro]

Does this mean that write access on files in /dev is limited just to
permissions change and, therefore, just some bytes of change in the
filesystem? This seems pretty much acceptable for me in terms of CF
wear-off, if it does not happen with a high frequence.


On Mon, Oct 19, 2015 at 12:27 PM, Stuart Henderson 
wrote:

>
> Some devices get chown()ed during normal system operation, see fbtab(5).

Re: Install on compact flash

[Stuart Henderson]

On 2015-10-19, Josh Grosse  wrote:
> On Mon, Oct 19, 2015 at 04:34:31AM +0200, Einfach Jemand wrote:
>>  No. As far as I understand it:
>> The type (char or block), the major and minor number of the device
>> special file and its name are means to activate the corresponding device
>> handler ("driver") in the kernel and the bytes are sent to the device
>> specified by the file. 
>
> Ok.  I can at least tell you that the last time I tested an r/o
> /dev was at OpenBSD 3.8 or so, and the filesystem was CD9660 rather
> than FFS.  
>
> It failed.  So from that point, until I stopped making live media
> images at 5.0, I never tested again. /dev was merely one of a half
> dozen r/w filesystems I used with MFS.

Some devices get chown()ed during normal system operation, see fbtab(5).

Re: Install on compact flash

[Bryan Irvine]

I ran native on compact flash as an experiment for 5+ years without ever
changing the CF card. I only migrated away from it because my old soekris
couldn't keep up with my internet speeds once I upgraded. It still boots
and works fine. Personally I found the hassle of maintaining a ramdisk
frankenstein was worse than just replacing the compact flash should it fail
(which it never did.) but of course YMMV.

On Thu, Oct 15, 2015 at 9:19 AM, Paolo Aglialoro  wrote:

> Hello,
>
> I would like to create an embedded amd64 installation, with system running
> on a 8GB 233x CF card attached to an Intel ITX mb.
>
> In order to minimise nand wear off, I would like to put on ramdisk (the
> machine would have 2GB ram, so I believe enough also for that, but I still
> can upgrade it to 4GB if needed) the parts of the file hierarchy with most
> intensive system write I/O, like, for instance, /tmp and I imagine some
> parts of /var.
>
> My questions are two:
> 1. What are the dirs I should take into account to go to ramdisk?
> 2. What is the correct filesystem type to put in fstab for all the entries
> of point 1. in order to store them in ramdisk?
>
> Thanks in advance for ur answers

Re: Install on compact flash

[Philip Guenther]

On Mon, Oct 19, 2015 at 5:02 AM, Stuart Henderson  wrote:
> On 2015-10-19, Paolo Aglialoro  wrote:
>> On Mon, Oct 19, 2015 at 12:27 PM, Stuart Henderson  
>> wrote:
>>
>>> Some devices get chown()ed during normal system operation, see fbtab(5).
>>
>> Does this mean that write access on files in /dev is limited just to
>> permissions change and, therefore, just some bytes of change in the
>> filesystem? This seems pretty much acceptable for me in terms of CF
>> wear-off, if it does not happen with a high frequence.
>
> Timestamps are updated as well. But I don't think wear-off is really
> a problem for any kind of normal use. It's certainly less of a problem
> than the need to handle a separate /dev partition and make sure that
> things are kept in-sync through OS updates.

Note that FFS, when not mounted with softdeps, will delay writing out
inode updates *for device files* when all that's being updated is the
timestamps.  They get flushed to disk when the vnode is reclaimed,
which is probably only when the filesystem is unmounted when shutting
down.  That was a nice little enhancement we merged from FreeBSD over
a year ago.


Philip Guenther

Re: Install on compact flash

[Einfach Jemand]

Am 18.10.2015 um 15:50 schrieb Josh Grosse:
> On Sun, Oct 18, 2015 at 01:08:48PM +0200, Paolo Aglialoro wrote:
> 
>> ...Still cannot understand why some of you have adviced to put /dev on
>> ramdisk: isn't MAKEDEV just run at install time and then only manually?
>> From what emerged in the discussion it looks that, besides /home, /var is
>> the only place where system makes its usual writes.
> 
> The "some of you" was me, if I recall the complete discussion thread.  :)
> 
> While it's true MAKEDEV needs to create the device special files, the
> device special files that are stored there need to be writeable.  
> 
> Here's one key example:
> 
> You mount(8) a device special file when you mount a filesystem.  
> All write operations to that mounted filesystem are directed through 
> the device special file.
>

No. As far as I understand it:
The type (char or block), the major and minor number of the device
special file and its name are means to activate the corresponding device
handler ("driver") in the kernel and the bytes are sent to the device
specified by the file. A simple test: If your /dev filesystem resides on
wd0a and you write from the network to a mounted filesystem on a second
disc, e.g. /dev/wd1a on /mnt and you watch this with iostat you will see
almost no activity on wd0.

To quote from "D.M. Ritchie and K. Thompson: ``The UNIX Time-Sharing
System'' Communications of the ACM, Vol. 17, No. 7, July 1974.":

"Special files constitute the most unusual feature of the UNIX
filesystem. Each supported I/O device is associated with at least one
such file. Special files are read and written just like ordinary disk
files, but requests to read or write result in activation of the
associated device. An entry for each special file resides in directory
/dev, although a link may be made to one of these files just as it may
to an ordinary file. Thus, for example, to write on a magnetic tape one
may write on the file /dev/mt. Special files exist for each
communication line, each disk, each tape drive, and for physical main
memory. Of course, the active disks and the memory special files are
protected from indiscriminate access. There is a threefold advantage in
treating I/O devices this way: file and device I/O are as similar as
possible; file and device names have the same syntax and meaning, so
that a program expecting a file name as a parameter can be passed a
device name; finally, special files are subject to the same protection
mechanism as regular files."

So I see no harm in /dev residing on the flash device.

> 
> 
> While you won't run MAKEDEV while the system is up and running,
> take a look at the MAKEDEV(8) man page.  You'll see a lot of device
> special files that will need to be writeable.
> 

Yes, but this is an _attribute_ of the device file that is used by the
kernel and the device handler. Nothing gets written to a device file in
the sense bytes are written to an ordinary file in a fileystem (e.g.,
/tmp/test).

HTH
rru

Re: Install on compact flash

[Josh Grosse]

On Mon, Oct 19, 2015 at 04:34:31AM +0200, Einfach Jemand wrote:
>  No. As far as I understand it:
> The type (char or block), the major and minor number of the device
> special file and its name are means to activate the corresponding device
> handler ("driver") in the kernel and the bytes are sent to the device
> specified by the file. 

Ok.  I can at least tell you that the last time I tested an r/o
/dev was at OpenBSD 3.8 or so, and the filesystem was CD9660 rather
than FFS.  

It failed.  So from that point, until I stopped making live media
images at 5.0, I never tested again. /dev was merely one of a half
dozen r/w filesystems I used with MFS.

Re: Install on compact flash

[Josh Grosse]

On Sun, Oct 18, 2015 at 01:08:48PM +0200, Paolo Aglialoro wrote:

> ...Still cannot understand why some of you have adviced to put /dev on
> ramdisk: isn't MAKEDEV just run at install time and then only manually?
> From what emerged in the discussion it looks that, besides /home, /var is
> the only place where system makes its usual writes.

The "some of you" was me, if I recall the complete discussion thread.  :)

While it's true MAKEDEV needs to create the device special files, the
device special files that are stored there need to be writeable.  

Here's one key example:

You mount(8) a device special file when you mount a filesystem.  
All write operations to that mounted filesystem are directed through 
the device special file.



While you won't run MAKEDEV while the system is up and running,
take a look at the MAKEDEV(8) man page.  You'll see a lot of device
special files that will need to be writeable.

Re: Install on compact flash

[Paolo Aglialoro]

Hi folks,

first of all thanks for all your interesting answers! I didn't expect this
topic to be so popular and I am quite satisfied to have received a broader
perspective than I had in mind.

My fault was not specifying exactly what jobs the machine would perform,
that could have narrowed down the possible answers. This machine is a
special project as, in itself is both a mod and a low power box. It's
purpose will be to be a "near-TV-set" server/client, doing stuff like
torrent, sftp, ftp, dlna, web surfing, media playing, gns3, etc. It should
use E17 as X desktop environment.

The original idea was that it had to be at the same time:
- small enough to fit a living room without being noticed
- not power hungry (very likely to be often 24h on)
- kinda stylish

The stylish hull came from a dead SparcClassic which has been completely
overhauled internally to fit a miniITX board and externally (painted in
lucid black, original leds, metal plates with "Sun" and "SparcClassic"
preserved), instead of a floppy drive it now features  a CF reader which
gives you the ability of "taking away the OS with you" whenever you want
like the keys of a car.

The box is sleek enough to fit any living room being woman-scream proof and
at the same time capable of potentially hosting two HD, one 5" 1/4 and one
3" 1/2.

After having read all the answers, I have excluded options which are read
only or modify the system so much that render it unsupported in a major
way, making also upgrades like gambling. This machine will be running
-release.

What I had more or less in mind, which buys the ticket, is what Marko and
Kenneth have suggested, so using mfs_mount with -P and rsyncing at shutdown
or even at regular intervals. I see this as the best compromise between a
totally ro system with unsupported upgrades and a bare CF- install without
being certain of timings for media wear-off. I would also like to minimise
the CF-upgrade process through time: it is time consuming and, with a
consistent policy to reduce CF writes, likely unnecessary.

The solution will look like the following:

- CF for / with mfs_mount -P for all /var
- 5" 1/4 HD for /home and swap
- 3" 1/2 HD somewhere inder /mnt for write intensive jobs, like for
instance torrent
- an extra rpm-regulated case side fan to exhaust extra heat if present

Still cannot understand why some of you have adviced to put /dev on
ramdisk: isn't MAKEDEV just run at install time and then only manually?
>From what emerged in the discussion it looks that, besides /home, /var is
the only place where system makes its usual writes.

Btw, thanks for the suggestions about reconciling syslog with security
checks!

Looks like I have the instruments to proceed now, pictures of the
SparcClassic mod are available in private for the curious ones.

Thanks!!!

Re: Install on compact flash

[Craig Skinner]

On 2015-10-15 Thu 16:01 PM |, Kenneth Gober wrote:
> 
> You may also want to move cron log files from /var/cron to /var/log (or
> /var/log/cron if you want them to be separated from other logs).  Doing so
> involves modifying /etc/syslog.conf and /etc/newsyslog.conf.  If you do
> this be prepared for warnings in the nightly security check emails due to the
> cron log file not being found in its normal location.  Leaving an empty log
> file in the original location is one workaround for this but I haven't tried
> it.

That stops the security alerts.

>  A better workaround may be to modify the nightly check so that it looks
> for the log file in its new location.
> 

Relocate the line in /etc/mtree/special
-- 
The Heineken Uncertainty Principle:
You can never be sure how many beers you had last night.

Re: Install on compact flash

[Nick Holland]

On 10/15/15 12:43, Josh Grosse wrote:
> On 2015-10-15 12:19, Paolo Aglialoro wrote:
> 
> MyTL;DR - Don't bother.  I used to create "live media" read/only 
> systems,
> with MFS filesystems for read/write.
> 
> You'll find maintaining a read-only system much more difficult.

exactly.

There are a lot of reasons floating gate technology devices (that would
be "Flash", "EEPROM". "EPROM" and kin) fail, ONE of them is write
fatigue.  Obsessing over this ONE failure mode is silly...and in my
experience (like Stuart's) NOT the right thing.

Kinda like obsessing over the tires on your car failing due to the tread
wearing out...and ignoring the nails in the road, potholes, bomb craters
and other such things that will interrupt your travel faster and with
less warning.

Worried about write fatigue?  ok, replace your flash device every five
years or so.  And have a plan in place for all the other things that
/will/ take you down much more often.  Or don't worry about the write
fatigue and replace it or whatever else fails whenever needed.  Or when
it is comically obsolete.

Again, a case of optimizing the wrong thing.  By "improving" your system
as you propose, you will almost certainly create far more downtime and
work on your part.

Nick.

Re: Install on compact flash

[Jan Stary]

On Oct 15 18:19:12, paol...@gmail.com wrote:
> I would like to create an embedded amd64 installation, with system running
> on a 8GB 233x CF card attached to an Intel ITX mb.
> 
> In order to minimise nand wear off, I would like to put on ramdisk (the
> machine would have 2GB ram, so I believe enough also for that, but I still
> can upgrade it to 4GB if needed) the parts of the file hierarchy with most
> intensive system write I/O, like, for instance, /tmp and I imagine some
> parts of /var.
> 
> My questions are two:
> 1. What are the dirs I should take into account to go to ramdisk?
> 2. What is the correct filesystem type to put in fstab for all the entries
> of point 1. in order to store them in ramdisk?

You give no information about what the machine is gonna do.
Do you actually expect "intensive I/O" in /tmp or /var?

I have been running my home router/dns/mx out of a 4GB CF card
since 2009, treating it as any other installation,
and never had a single problem.

Unless you have a specific reason not to,
treat your CF card as any other disk.

Jan

Re: Install on compact flash

[Kenneth Gober]

On Thu, Oct 15, 2015 at 12:19 PM, Paolo Aglialoro  wrote:
> In order to minimise nand wear off, I would like to put on ramdisk (the
> machine would have 2GB ram, so I believe enough also for that, but I still
> can upgrade it to 4GB if needed) the parts of the file hierarchy with most
> intensive system write I/O, like, for instance, /tmp and I imagine some
> parts of /var.
>
> My questions are two:
> 1. What are the dirs I should take into account to go to ramdisk?
> 2. What is the correct filesystem type to put in fstab for all the entries
> of point 1. in order to store them in ramdisk?

I noticed that you have not said that you would like your other filesystems
to be read-only, only that you want the 'most intensive' writes to go to a
ramdisk.  In other words, you don't want your flash storage to be read-only,
you merely want it to be read-mostly.

I have done something similar myself and what I did was to make 2 mfs
filesystems: /tmp and /var/log.  here are my /etc/fstab entries:

  swap /tmp mfs rw,nodev,nosuid,-s=128M 0 0
  swap /var/log mfs rw,nodev,nosuid,-s=128M,-P=/mfs/log 0 0

Note that in my setup I have an /mfs/log partition.  This was originally the
/var/log partition from installation, which I renamed in /etc/fstab later
(after doing mkdir -p /mfs/log first).  This is a 'durable' copy of /var/log
meant to survive crashes, reboots or power failures.  The contents of
/var/log are periodically checkpointed to /mfs/log using rsync (installed
from packages), via a cron job:

  # checkpoint log files
  10 */4 * * * /usr/local/bin/rsync -aH --delete /var/log/ /mfs/log

Also, I added this line to /etc/rc.shutdown to handle reboots:

  /usr/local/bin/rsync -aH --delete /var/log/ /mfs/log

If /var/tmp isn't already a symbolic link to /tmp, do that as well to
redirect another commonly written directory to ramdisk.

You may also want to move cron log files from /var/cron to /var/log (or
/var/log/cron if you want them to be separated from other logs).  Doing so
involves modifying /etc/syslog.conf and /etc/newsyslog.conf.  If you do
this be prepared for warnings in the nightly security check emails due to the
cron log file not being found in its normal location.  Leaving an empty log
file in the original location is one workaround for this but I haven't tried
it.  A better workaround may be to modify the nightly check so that it looks
for the log file in its new location.

-ken

Re: Install on compact flash

[Josh Grosse]

On 2015-10-15 12:19, Paolo Aglialoro wrote:

MyTL;DR - Don't bother.  I used to create "live media" read/only 
systems,

with MFS filesystems for read/write.

You'll find maintaining a read-only system much more difficult.

I recommend just treating your flash memory like a disk drive.  Modern
wear leveling technology means read-only CF devices are no longer
a necessity.  I run several routers with CF cards, and all filesystems
are read/write.

As to your questions:


... 1. What are the dirs I should take into account to go to ramdisk?


/dev, /etc, /var, and /tmp require write capability, and while they
do not require it, consider adding /root and /home to list of writeable
filesystems.

2. What is the correct filesystem type to put in fstab for all the 
entries

of point 1. in order to store them in ramdisk?


See the -P option of mount_mfs(8)

Install on compact flash

[Paolo Aglialoro]

Hello,

I would like to create an embedded amd64 installation, with system running
on a 8GB 233x CF card attached to an Intel ITX mb.

In order to minimise nand wear off, I would like to put on ramdisk (the
machine would have 2GB ram, so I believe enough also for that, but I still
can upgrade it to 4GB if needed) the parts of the file hierarchy with most
intensive system write I/O, like, for instance, /tmp and I imagine some
parts of /var.

My questions are two:
1. What are the dirs I should take into account to go to ramdisk?
2. What is the correct filesystem type to put in fstab for all the entries
of point 1. in order to store them in ramdisk?

Thanks in advance for ur answers

Re: Install on compact flash

[Stefan Sperling]

On Thu, Oct 15, 2015 at 06:19:12PM +0200, Paolo Aglialoro wrote:
> 2. What is the correct filesystem type to put in fstab for all the entries
> of point 1. in order to store them in ramdisk?

I'm using a line such as:

swap /var/log mfs rw,nodev,nosuid,-s40M,-P/var/log.tmpl 0 0

to put /var/log into RAM. See man mount_mfs for mount options.

The /var/log.tmpl directory is a template which I simply renamed from
the original /var/log directory. This keeps permissions intact and
ensures that required files are present.

As an alternative, there's also tmpfs (see man mount_tmpfs).
The fstab line looks almost the same:

swap /ramdisk tmpfs rw,nodev,nosuid,-s128M 0 0

Re: Install on compact flash

[Jan Vlach]

Hi Paolo,

check out flashrd as a starting point (possibly also finishing) - 
http://www.nmedia.net/flashrd/ 

Jan

Re: Install on compact flash

[Noah]

I run NetBSD 7.0/hpcarm (previously NBSD 6.1.1 through 6.1.5) on a
daily-use machine directly from a CF since August 2013. During an upgrade
from 6.1.5 to NetBSD 7.0 RC2 a few months ago, I finally ran into a pretty
gnarly write error condition on a 4GB SanDisk CF. I bought a fresh 16GB
card to replace it, but formatting it seems to have worked (fingers
crossed) and it's still working fine in a different machine.  CF is pretty
robust. I would hesitate to put a swap partition on it, but other than
that, I'd think you would be fine, especially with syslog buffering in RAM
as Stu mentioned.

Back in 2006, I was running a cluster of 200-300 OpenBSD 4.x 1U rackmounts
purely from ramdisk, booting from 64MB Kingston USB sticks. I won't lie. It
was kind of a pain in the butt, but things are probably a little better
these days.

On Thu, Oct 15, 2015 at 1:57 PM, Stuart Henderson 
wrote:

> On 2015-10-15, Stefan Sperling  wrote:
> > On Thu, Oct 15, 2015 at 06:19:12PM +0200, Paolo Aglialoro wrote:
> >> 2. What is the correct filesystem type to put in fstab for all the
> entries
> >> of point 1. in order to store them in ramdisk?
> >
> > I'm using a line such as:
> >
> > swap /var/log mfs rw,nodev,nosuid,-s40M,-P/var/log.tmpl 0 0
> >
> > to put /var/log into RAM. See man mount_mfs for mount options.
> >
> > The /var/log.tmpl directory is a template which I simply renamed from
> > the original /var/log directory. This keeps permissions intact and
> > ensures that required files are present.
> >
> > As an alternative, there's also tmpfs (see man mount_tmpfs).
> > The fstab line looks almost the same:
> >
> > swap /ramdisk tmpfs rw,nodev,nosuid,-s128M 0 0
> >
> >
>
> Or you can use 'syslogd -s' and memory buffer logging (syslogc to view
> them). The main reason I'd see for doing this is to reduce the risk of
> dirty filesystems getting in the way of a reboot. CFs can handle plenty
> of writes - I've personally used CF cards to run systems on for quite
> a few years - I've seen a few failures relatively early in their life
> (including ones with very writes) but don't remember any failing later
> in any way that could be attributed to writes - though I've retired
> quite a few due to them being too small after a couple of years of
> updates (too much surgery is required to run a recent OS on a 64MB
> card ;)

Re: Install on compact flash

[Marko Cupać]

On Thu, 15 Oct 2015 18:19:12 +0200
Paolo Aglialoro  wrote:

> Hello,
>
> I would like to create an embedded amd64 installation, with system
> running on a 8GB 233x CF card attached to an Intel ITX mb.
>
> In order to minimise nand wear off, I would like to put on ramdisk
> (the machine would have 2GB ram, so I believe enough also for that,
> but I still can upgrade it to 4GB if needed) the parts of the file
> hierarchy with most intensive system write I/O, like, for
> instance, /tmp and I imagine some parts of /var.
>
> My questions are two:
> 1. What are the dirs I should take into account to go to ramdisk?
> 2. What is the correct filesystem type to put in fstab for all the
> entries of point 1. in order to store them in ramdisk?
>
> Thanks in advance for ur answers
>

1. /var and /dev
2. swap /var mfs rw,-P=/mfs/var,-s=32768,nodev,nosuid,noexec 0 0
   swap /dev mfs rw,-P=/mfs/dev,-s=8192,-i=128,nosuid,noexec 0 0

Back in time when I was building OpenBSD system resistant to frequent
power outages for the first time (completely read-only), I created this
'works-for-me' howto:
https://www.mimar.rs/sysadmin/2014/openbsd-otporan-na-nestanke-struje

It is in Serbian, but most important stuff is code and commands, the
rest will probably be understandable by help of some online translator.

Regards,
--
Before enlightenment - chop wood, draw water.
After  enlightenment - chop wood, draw water.

Marko Cupać
https://www.mimar.rs/

Re: Install on compact flash

[Stuart Henderson]

On 2015-10-15, Stefan Sperling  wrote:
> On Thu, Oct 15, 2015 at 06:19:12PM +0200, Paolo Aglialoro wrote:
>> 2. What is the correct filesystem type to put in fstab for all the entries
>> of point 1. in order to store them in ramdisk?
>
> I'm using a line such as:
>
> swap /var/log mfs rw,nodev,nosuid,-s40M,-P/var/log.tmpl 0 0
>
> to put /var/log into RAM. See man mount_mfs for mount options.
>
> The /var/log.tmpl directory is a template which I simply renamed from
> the original /var/log directory. This keeps permissions intact and
> ensures that required files are present.
>
> As an alternative, there's also tmpfs (see man mount_tmpfs).
> The fstab line looks almost the same:
>
> swap /ramdisk tmpfs rw,nodev,nosuid,-s128M 0 0
>
>

Or you can use 'syslogd -s' and memory buffer logging (syslogc to view
them). The main reason I'd see for doing this is to reduce the risk of
dirty filesystems getting in the way of a reboot. CFs can handle plenty
of writes - I've personally used CF cards to run systems on for quite
a few years - I've seen a few failures relatively early in their life
(including ones with very writes) but don't remember any failing later
in any way that could be attributed to writes - though I've retired
quite a few due to them being too small after a couple of years of
updates (too much surgery is required to run a recent OS on a 64MB
card ;)