Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-12 Thread Lampshade
> Hi!
>
> If I understand mount(8) (http://man.openbsd.org/mount) right, FFS
> mounts have a metadata I/O mode and a data I/O mode. By default,
> metadata is accessed synchronously and data is accessed
> asynchronously.
>
> "-o sync" will force both to synchronous mode, and "-o softdep" would
> change the metadata I/O mode to the alternative softdep access mode.

No. softdep and async are different concepts.
Default: metadata, data accordingly: sync, async
not that there is async - not softdep

You can do the oposite by options: nosync, noasync
It would mean asynchronous mode for metadata and synchronous for data - which 
is stupid (slow and dangerous), so don't.

I think default are quite good for SSD. Maybe add noatime for some partitions.



Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-11 Thread Tom Smyth
Hi Tinker,

If you don't expect to run into unexpected shutdowns (power .. you will
be ok with defaults ) ie if you believe the system will always
have reliable power, (and you will have access to the console to help
fsck on reboot, in the unlikely event of loss of power to the running system)

the other thing I would suggest to you (forgot to mention)
is modify the default offset of the first partition from 64 sectors
to 2048 sectors (1 MByte)
and size the rest of the partitions in sectors so that the number you
enter is divisible evenly by 2048, this will (hopefully)  mean that allocation
units on the filesystem are aligned with the underlying SSD disk
which are usually split in to 1MB blocks

I Hope this helps

Thanks
Tom Smyth

On 11 February 2018 at 05:06, Tinker  wrote:
> My original question was which mounting options are optimal for FFS
> filesystems on SSD:s, for data-safety and relative IO speed.
>
> It seems noone in this thread had any data-safety issues ever and so
> there was no feedback beyond "use the defaults".
>
> I guess "noatime" as a measure to lower write load should be of some
> utility for data-safety.
>
>
> Part of my question with this thread was if the "softdep" algorithm
> used could be expected to be more data-safe for filesystem metadata
> than the normal "sync" algorithm, for instance by appending rather
> than overwriting.
>
>
> Tinker
>



-- 
Kindest regards,
Tom Smyth

Mobile: +353 87 6193172
The information contained in this E-mail is intended only for the
confidential use of the named recipient. If the reader of this message
is not the intended recipient or the person responsible for
delivering it to the recipient, you are hereby notified that you have
received this communication in error and that any review,
dissemination or copying of this communication is strictly prohibited.
If you have received this in error, please notify the sender
immediately by telephone at the number above and erase the message
You are requested to carry out your own virus check before
opening any attachment.



Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-11 Thread Marcus MERIGHI
t1...@protonmail.ch (Tinker), 2018.02.11 (Sun) 06:06 (CET):
> My original question was which mounting options are optimal for FFS
> filesystems on SSD:s, for data-safety and relative IO speed.
> 
> It seems noone in this thread had any data-safety issues ever and so
> there was no feedback beyond "use the defaults".
> 
> I guess "noatime" as a measure to lower write load should be of some
> utility for data-safety.
> 
> 
> Part of my question with this thread was if the "softdep" algorithm
> used could be expected to be more data-safe for filesystem metadata
> than the normal "sync" algorithm, for instance by appending rather
> than overwriting.

what I found yesterday after reading your questions and not replying...

ratchov@
https://marc.info/?l=openbsd-misc=142166078120639
nick@
https://marc.info/?l=openbsd-misc=142170287802566
schwarze@
https://marc.info/?l=openbsd-misc=142204644507870
tedu@
https://marc.info/?l=openbsd-misc=142250784228719
https://marc.info/?l=openbsd-misc=142294185000751
https://marc.info/?l=openbsd-tech=143770961724850
clint pachl
https://marc.info/?l=openbsd-misc=142294090200592
sthen@
https://marc.info/?l=openbsd-tech=143774281500609

Marcus



Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-11 Thread Tinker
My original question was which mounting options are optimal for FFS
filesystems on SSD:s, for data-safety and relative IO speed.

It seems noone in this thread had any data-safety issues ever and so
there was no feedback beyond "use the defaults".

I guess "noatime" as a measure to lower write load should be of some
utility for data-safety.


Part of my question with this thread was if the "softdep" algorithm
used could be expected to be more data-safe for filesystem metadata
than the normal "sync" algorithm, for instance by appending rather
than overwriting.


Tinker



Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-10 Thread Martin Schröder
2018-02-10 7:28 GMT+01:00 Rupert Gallagher :
> The only problem I've encountered is rsync unable to preserve the original 
> time of files: copied files have the time of the copy.

man rsync

-t, --times preserve modification times

You want
-a, --archive   archive mode; equals -rlptgoD (no -H,-A,-X)

Best
Martin



Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-10 Thread Rupert Gallagher
The only problem I've encountered is rsync unable to preserve the original time 
of files: copied files have the time of the copy.

Sent from ProtonMail Mobile

On Sat, Feb 10, 2018 at 01:50,  wrote:

>> From tom.sm...@wirelessconnect.eu Thu Feb 8 23:37:59 2018 > From: Tom Smyth  
>> > Date: Thu, 8 Feb 2018 19:40:23 + > Subject: Re: For a FFS on an SSD, 
>> which of "-o" nil, "sync" &/ "softdep" is > more data-safe and fast? > To: 
>> Tinker  > > Also use noatime mount option so whe reading files you are not 
>> updating > access time But then don't complain when your favorite software 
>> package either doesn't work or does unexpected things. Martin 
>> @protonmail.ch> @wirelessconnect.eu>


Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-10 Thread Ax0n
I use ls -lu and find -atime quite frequently. I've also been using OpenBSD
as a desktop (with all the insane disk activity of a browser cache, temp
files, git, etc) while doing absolutely nothing special using SATA SSD
drives. I'm running Softraid crypto on one, running without softraid on the
other. No snazzy mount options outside the defaults. It's been as reliable
as any spinning rust I've used. You're keeping good backups regardless,
right? Because that protects you from many more likely failure modes than
wear-level-exhaustion on SSD.

On Sat, Feb 10, 2018 at 10:38 AM,  wrote:

> > From tom.sm...@wirelessconnect.eu Sat Feb 10 11:28:46 2018
> > From: Tom Smyth 
> > Date: Sat, 10 Feb 2018 09:10:30 +
> > Subject: Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep"
> is
> >  more data-safe and fast?
> > To: mar...@martinbrandenburg.com
> >
> > Hi Martin... can you give a specific case where you have experienced
> > negative impacts from thevmount options i suggested...
> > It would be good to know...
>
> Well I don't run noatime, so I can't give specific examples.
>
> The point I was making is that noatime is a tradeoff.  You give up
> adherence to POSIX and historical Unix behavior in exchange for
> performance and less disk wear.
>
> Now I will admit that programs that rely on atime updates are few and
> far between.  It was always a bit niche.  Nowadays, most software is
> written on Linux, and most Linux systems use relatime.
>
> It might be an acceptable tradeoff.  To use the examples from the
> manpage, if you know you're not going run any programs that need it and
> that you'll never leave your laptop up for seven days or if you know
> your news server doesn't need it, go ahead and disable it.
>
> Just don't forget to consider it as a potential problem if you run into
> trouble.
>
> Here are some lines from /etc/daily.  Without atime updates, it could
> delete files.  (Some of these might get more frequent mtime or ctime
> updates which would bump the atime under noatime.)
>
> The second set might be uncommented on a big multiuser system to clear
> out /tmp since users don't.  They might be a little irritated when the
> file they've been reading for the last day gets deleted.
>
> next_part "Removing scratch and junk files:"
> if [ -d /tmp -a ! -L /tmp ]; then
> cd /tmp && {
> find -x . \
> \( -path './ssh-*' -o -path ./.X11-unix -o -path ./.ICE-unix \
> -o -path './tmux-*' \) \
> -prune -o -type f -atime +7 -execdir rm -f -- {} \; 2>/dev/null
> find -x . -type d -mtime +1 ! -path ./vi.recover ! -path
> ./.X11-unix \
> ! -path ./.ICE-unix ! -name . \
> -execdir rmdir -- {} \; >/dev/null 2>&1; }
> fi
>
> # Additional junk directory cleanup would go like this:
> #if [ -d /scratch -a ! -L /scratch ]; then
> #   cd /scratch && {
> #   find . ! -name . -atime +1 -execdir rm -f -- {} \;
> #   find . ! -name . -type d -mtime +1 -execdir rmdir -- {} \; \
> #   >/dev/null 2>&1; }
> #fi
>
> Martin
>
>


Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-10 Thread martin
> From tom.sm...@wirelessconnect.eu Sat Feb 10 11:28:46 2018
> From: Tom Smyth 
> Date: Sat, 10 Feb 2018 09:10:30 +
> Subject: Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is
>  more data-safe and fast?
> To: mar...@martinbrandenburg.com
>
> Hi Martin... can you give a specific case where you have experienced
> negative impacts from thevmount options i suggested...
> It would be good to know...

Well I don't run noatime, so I can't give specific examples.

The point I was making is that noatime is a tradeoff.  You give up
adherence to POSIX and historical Unix behavior in exchange for
performance and less disk wear.

Now I will admit that programs that rely on atime updates are few and
far between.  It was always a bit niche.  Nowadays, most software is
written on Linux, and most Linux systems use relatime.

It might be an acceptable tradeoff.  To use the examples from the
manpage, if you know you're not going run any programs that need it and
that you'll never leave your laptop up for seven days or if you know
your news server doesn't need it, go ahead and disable it.

Just don't forget to consider it as a potential problem if you run into
trouble.

Here are some lines from /etc/daily.  Without atime updates, it could
delete files.  (Some of these might get more frequent mtime or ctime
updates which would bump the atime under noatime.)

The second set might be uncommented on a big multiuser system to clear
out /tmp since users don't.  They might be a little irritated when the
file they've been reading for the last day gets deleted.

next_part "Removing scratch and junk files:"
if [ -d /tmp -a ! -L /tmp ]; then
cd /tmp && {
find -x . \
\( -path './ssh-*' -o -path ./.X11-unix -o -path ./.ICE-unix \
-o -path './tmux-*' \) \
-prune -o -type f -atime +7 -execdir rm -f -- {} \; 2>/dev/null
find -x . -type d -mtime +1 ! -path ./vi.recover ! -path ./.X11-unix \
! -path ./.ICE-unix ! -name . \
-execdir rmdir -- {} \; >/dev/null 2>&1; }
fi

# Additional junk directory cleanup would go like this:
#if [ -d /scratch -a ! -L /scratch ]; then
#   cd /scratch && {
#   find . ! -name . -atime +1 -execdir rm -f -- {} \;
#   find . ! -name . -type d -mtime +1 -execdir rmdir -- {} \; \
#   >/dev/null 2>&1; }
#fi

Martin



Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-10 Thread Tom Smyth
Hi Martin... can you give a specific case where you have experienced
negative impacts from thevmount options i suggested...
It would be good to know...

Thanks Martin

Tom Smyth

On 10 Feb 2018 12:50 AM,  wrote:

> > From tom.sm...@wirelessconnect.eu Thu Feb  8 23:37:59 2018
> > From: Tom Smyth 
> > Date: Thu, 8 Feb 2018 19:40:23 +
> > Subject: Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep"
> is
> >  more data-safe and fast?
> > To: Tinker 
> >
> > Also use  noatime  mount option so whe reading files you are not updating
> > access time
>
> But then don't complain when your favorite software package either
> doesn't work or does unexpected things.
>
> Martin
>


Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-09 Thread martin
> From tom.sm...@wirelessconnect.eu Thu Feb  8 23:37:59 2018
> From: Tom Smyth 
> Date: Thu, 8 Feb 2018 19:40:23 +
> Subject: Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is
>  more data-safe and fast?
> To: Tinker 
>
> Also use  noatime  mount option so whe reading files you are not updating
> access time

But then don't complain when your favorite software package either
doesn't work or does unexpected things.

Martin



Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-08 Thread Tom Smyth
Also use  noatime  mount option so whe reading files you are not updating
access time

Ie there would be writes to disk everytime u access a file if noatime is
not set



On 8 Feb 2018 7:36 PM, "Tinker"  wrote:

> Hi!
>
> If I understand mount(8) (http://man.openbsd.org/mount) right, FFS
> mounts have a metadata I/O mode and a data I/O mode. By default,
> metadata is accessed synchronously and data is accessed
> asynchronously.
>
> "-o sync" will force both to synchronous mode, and "-o softdep" would
> change the metadata I/O mode to the alternative softdep access mode.
>
>
> What is the optimal sync &/ softdep configuration for minimizing risk
> of data loss on unexpected reboot, and maximizing overall filesystem
> I/O speed?
>
> By "data loss" I mean in particular filesystem structure issues that
> could cause fsck(8) hickups and stability issues (e.g. the "/var/log"
> directory disappearing altogether - I've seen that happen), secondarily
> that file data content updates not would have stuck on the disk.
>
> Either nothing (no "-o" setting), or "-o sync,softdep", or "-o sync",
> would all look like they'd make sense.
>
> (Can you even do "-o sync,softdep", would that serve any purpose?)
>
> Will appreciate to learn your thoughts on this one a lot,
>
> Thanks,
> Tinker
>
>


Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-08 Thread Tom Smyth
Also use  noatime  mount option so whe reading files you are not updating
access time

On 8 Feb 2018 7:36 PM, "Tinker"  wrote:

> Hi!
>
> If I understand mount(8) (http://man.openbsd.org/mount) right, FFS
> mounts have a metadata I/O mode and a data I/O mode. By default,
> metadata is accessed synchronously and data is accessed
> asynchronously.
>
> "-o sync" will force both to synchronous mode, and "-o softdep" would
> change the metadata I/O mode to the alternative softdep access mode.
>
>
> What is the optimal sync &/ softdep configuration for minimizing risk
> of data loss on unexpected reboot, and maximizing overall filesystem
> I/O speed?
>
> By "data loss" I mean in particular filesystem structure issues that
> could cause fsck(8) hickups and stability issues (e.g. the "/var/log"
> directory disappearing altogether - I've seen that happen), secondarily
> that file data content updates not would have stuck on the disk.
>
> Either nothing (no "-o" setting), or "-o sync,softdep", or "-o sync",
> would all look like they'd make sense.
>
> (Can you even do "-o sync,softdep", would that serve any purpose?)
>
> Will appreciate to learn your thoughts on this one a lot,
>
> Thanks,
> Tinker
>
>


For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

2018-02-08 Thread Tinker
Hi!

If I understand mount(8) (http://man.openbsd.org/mount) right, FFS
mounts have a metadata I/O mode and a data I/O mode. By default,
metadata is accessed synchronously and data is accessed
asynchronously.

"-o sync" will force both to synchronous mode, and "-o softdep" would
change the metadata I/O mode to the alternative softdep access mode.


What is the optimal sync &/ softdep configuration for minimizing risk
of data loss on unexpected reboot, and maximizing overall filesystem
I/O speed?

By "data loss" I mean in particular filesystem structure issues that
could cause fsck(8) hickups and stability issues (e.g. the "/var/log"
directory disappearing altogether - I've seen that happen), secondarily
that file data content updates not would have stuck on the disk.

Either nothing (no "-o" setting), or "-o sync,softdep", or "-o sync",
would all look like they'd make sense.

(Can you even do "-o sync,softdep", would that serve any purpose?)

Will appreciate to learn your thoughts on this one a lot,

Thanks,
Tinker