Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-05 Thread hiro
If HDDs weren't cheaper i would only use SSDs. and yes, the cheap
consumer stuff, like samsung 850 evo. Price is the only downside IMO.



Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-05 Thread hiro
often the usb3 hdd controllers are horrible, completely forgot about that :)



Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-04 Thread Bakul Shah
On Sun, 04 Feb 2018 11:02:57 +0100 hiro <23h...@gmail.com> wrote:
hiro writes:
> For home use a ZFS intent log and caches on a good 2,5" SSD in a
> battery-backed thinkpad seems like an easy, silent, fast and stable
> (even against data loss from power outage) basis, even if you only
> connect shitty USB3 HDD drives externally for the pools. Your data
> should be safe on the SSD as long as you make sure it's
> underprovisioned enough to make fair use of wear leveling for a long
> time.

Many of these USB3 HDDs have 1 year warranty for a reason.
SMART check on a few month old Seagate USB3 disk I have shows
it has already exceeded worst case of many parameters! It
still works so I can't return it.

You can buy 5 year warranty WD Gold or RE for decent prizes
and can put together your own disk array.

I don't use a separate ZFS intent log on my main fileserver as
I don't rely on NFS much (mostly just readonly access).  I
also don't use a laptops for fileservice as they don't have
ECC memory.



Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-04 Thread Digby R.S. Tarvin
I am not familiar with the kirkwoods that you mentioned.

Just to be clear, the USB drive I was describing is rotating media in an
external enclosure, not a memory stick. Generally self powered, as powering
a portable hard drive from USB with a RPi is asking for trouble.

I have stopped buying flash memory devices from eBay and other vendors that
are not well known with a reputation to protect - far to many counterfeits
with less storage than the packaging claims on the market, and if you are
unfortunate enough to try using them with the FAT or eFAT filesystem they
are supplied with, data will eventually wrap around and destroy itself,
probably after sufficient time that the seller (who may well be ignorant of
why  his/her stock was so cheap) is no longer around to complain to. A
minor annoyance for me, but must be a lot of unhappy people losing
irreplaceable photographs.. Fortunately for me, attempting for reformat
with a Linux filesystem tends to fail on such devices, so I find out
straight away and get to send them back and be refunded . I had to write a
dedicated test program to demonstrate the subterfuge on the original
filesystem - to prove that the reformatting was revealing an issue, not the
source of it (as was often claimed).

I also don't buy cheap USB stick.

I like these
https://www.kanguru.com/storage-accessories/kanguru-ss3.shtml
because they have a real hardware write protect capability. Indispensable
if you are going to be inserting them into other people's machines, but
surprisingly uncommon.
At over $200.00  for 256GB, they are are a bit upmarket, but I havn't had
any problem with them yet.

Anyway, so long as you are aware of the risks and limitations, flash memory
devices are a useful technology, but not a complete replacement for
rotating magnetic storage.

On 4 February 2018 at 21:46, hiro <23h...@gmail.com> wrote:

> Hey, thanks for explaining. this usage is surprisingly valid. I have
> some much much older kirkwoods for the same scenario. The benefit is:
> gigabit ethernet, higher stability, case included, power supply
> included (and no power problems as on rpi), lower price.
> I boot them all from USB HDDs, but I see how flash would save more
> power. Carry on! :)
>
> The main disagreement in this thread is calling all kinds of different
> flash storage "SSD". common usage reserves this name for the sata or
> more recent nvme disks that actually are much more stable, in my
> understanding due to better controllers and their better wear leveling
> algorithms.
>
> With sd cards and usb flash drives you are lucky if your 128GB stick
> is not really 1GB flash with %1GB "wear leveling algorithm" where
> after 1GB you rewrite your already saved data :D
>
> It's a low-end market with shitty margins, low quality controllers,
> and in general too many counterfeits, even from good shops and big
> retailers. so you can't even depend on the company/brand/product name.
>
> Privately I never had surprising problems with HDDs, I don't manage to
> fill enough to notice the small risk in practice.
>
> All my old HDDs still work. They are only unused cause they got too
> small to be worth spinning any more (waste of power).
>
> I project my SSDs will not fail before i get 10/40gbit connection to
> my NAS. Till then my write wear will be limited by my low bandwidth
> and high latency practical use cases.
>
> On 2/4/18, Digby R.S. Tarvin  wrote:
> > static web pages, remote login (so that I can power/depower other
> hardware)
> > and file remote file distribution (via scp) mostly.
> >
> > The main requirement is very low standby power consumption so that it can
> > survive on batteries which are recharged using solar panels.
> >
> > Power consumption was the main reason for switching from laptops (~12W)
> to
> > Reaspberry Pis (1.2W)..
> >
> > The other advantage to the raspberry pi is that it is a cheap commodity
> > item - if I have one misbehave, I can just swap it out and throw it away.
> > However, other than failing SSD's, the raspberry Pis have proved very
> > reliable.
> >
> > On 4 February 2018 at 09:52, hiro <23h...@gmail.com> wrote:
> >
> >> > raspberry pi based servers
> >>
> >> what are you serving?
> >>
> >>
> >
>
>


Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-04 Thread Ole-Hjalmar Kristensen
The problem is the index. It is heavily updated, and I had a Fossil
installation that ate my SSD in about 6 months. The log was OK, so I could
rebuild the index on another disk.

On Sat, Feb 3, 2018 at 10:39 AM, lchg  wrote:

> As I know,  fossil/venti file system is log-structured,  so it may be good
> for flash devices, especially in extending life of flash devices.
>


Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-04 Thread hiro
Hey, thanks for explaining. this usage is surprisingly valid. I have
some much much older kirkwoods for the same scenario. The benefit is:
gigabit ethernet, higher stability, case included, power supply
included (and no power problems as on rpi), lower price.
I boot them all from USB HDDs, but I see how flash would save more
power. Carry on! :)

The main disagreement in this thread is calling all kinds of different
flash storage "SSD". common usage reserves this name for the sata or
more recent nvme disks that actually are much more stable, in my
understanding due to better controllers and their better wear leveling
algorithms.

With sd cards and usb flash drives you are lucky if your 128GB stick
is not really 1GB flash with %1GB "wear leveling algorithm" where
after 1GB you rewrite your already saved data :D

It's a low-end market with shitty margins, low quality controllers,
and in general too many counterfeits, even from good shops and big
retailers. so you can't even depend on the company/brand/product name.

Privately I never had surprising problems with HDDs, I don't manage to
fill enough to notice the small risk in practice.

All my old HDDs still work. They are only unused cause they got too
small to be worth spinning any more (waste of power).

I project my SSDs will not fail before i get 10/40gbit connection to
my NAS. Till then my write wear will be limited by my low bandwidth
and high latency practical use cases.

On 2/4/18, Digby R.S. Tarvin  wrote:
> static web pages, remote login (so that I can power/depower other hardware)
> and file remote file distribution (via scp) mostly.
>
> The main requirement is very low standby power consumption so that it can
> survive on batteries which are recharged using solar panels.
>
> Power consumption was the main reason for switching from laptops (~12W) to
> Reaspberry Pis (1.2W)..
>
> The other advantage to the raspberry pi is that it is a cheap commodity
> item - if I have one misbehave, I can just swap it out and throw it away.
> However, other than failing SSD's, the raspberry Pis have proved very
> reliable.
>
> On 4 February 2018 at 09:52, hiro <23h...@gmail.com> wrote:
>
>> > raspberry pi based servers
>>
>> what are you serving?
>>
>>
>



Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-04 Thread Digby R.S. Tarvin
static web pages, remote login (so that I can power/depower other hardware)
and file remote file distribution (via scp) mostly.

The main requirement is very low standby power consumption so that it can
survive on batteries which are recharged using solar panels.

Power consumption was the main reason for switching from laptops (~12W) to
Reaspberry Pis (1.2W)..

The other advantage to the raspberry pi is that it is a cheap commodity
item - if I have one misbehave, I can just swap it out and throw it away.
However, other than failing SSD's, the raspberry Pis have proved very
reliable.

On 4 February 2018 at 09:52, hiro <23h...@gmail.com> wrote:

> > raspberry pi based servers
>
> what are you serving?
>
>


Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-04 Thread hiro
For home use a ZFS intent log and caches on a good 2,5" SSD in a
battery-backed thinkpad seems like an easy, silent, fast and stable
(even against data loss from power outage) basis, even if you only
connect shitty USB3 HDD drives externally for the pools. Your data
should be safe on the SSD as long as you make sure it's
underprovisioned enough to make fair use of wear leveling for a long
time.



Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-04 Thread hiro
> raspberry pi based servers

what are you serving?



Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-04 Thread hiro
> uSD in raspberry pi

there's the error



Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-03 Thread Bakul Shah
On Sat, 03 Feb 2018 21:46:42 + "Digby R.S. Tarvin"  
wrote:
Digby R.S. Tarvin writes:
> 
>  Thats why I described my use case - to make the MTBF figures meaningful.  As
> I said, I have my system configured so that most heavy write accesses go to
> rotating media. I typically try to have my system partitions mounted read
> only, except var and tmp. I am currently using 32GB uSD devices for
> raspberry pi based servers, with about 100GB per year in writes (as
> reported by iostat), plus perhaps an initial 100GB in writing that occurs
> during installation and configuring. The last failure I had was about 3
> months ago, on uSD card that had been in use for just under 2 years.
>
> I can only speak of the experience that I have had, but I don't think my
> usage is sufficiently atypical to not count as 'in practice' for use as
> computer storage (ie I am not talking about  music players and cameras,
> phones etc).

Note that consumer grade uSD cards are much worse than SSDs --
they are mainly for the video recording market where capacity
is more important.  Not to mention The RasPis are poor at
reliability.  Even a xenon flash or near a RasPi could power a
RasPi2 down! And since they do no onboard power regulation,
people had lots of problems early on -- add one more USB
device and the thing can become unreliable.

At 100GB per year the SSD I am talking about should
last 3000 years!

Newer *high endurance* SanDisk microSDXC cards for example can
do 10k hours of video recording at a rate of 26Mbps speed.
That translates to about 117TBW. At 100GB writes/year they
should last over 1000 years.

> So from my experience, I would still tend to go along with Erik's advice
> (as relayed by Steve), or perhaps be even more fastidious about backups
> when using flash...

I think this has shifted more in favor of SSDs in the past
couple years.

The convention wisdom for ZFS was to not use SSDs for its
intent log (as all writes go through it) but it was okay to
use SSDs for the read cache, L2ARC (though that may not be
very effective). This is similar to Erik's recommendation.
But this falls apart if you use SSDs as your main storage.

To use a 300TBW device for 10 years, you'd have to *average*
over 82GB a day per device. I certainly don't see traffic
anywhere near that. If it weren't for the cost, I would
rebuilt my zfs array using SSDs. [Note that ZFS does its own
checksumming and has redundancy of its own]



Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-03 Thread Digby R.S. Tarvin
 Thats why I described my use case - to make the MTBF figures meaningful.  As
I said, I have my system configured so that most heavy write accesses go to
rotating media. I typically try to have my system partitions mounted read
only, except var and tmp. I am currently using 32GB uSD devices for
raspberry pi based servers, with about 100GB per year in writes (as
reported by iostat), plus perhaps an initial 100GB in writing that occurs
during installation and configuring. The last failure I had was about 3
months ago, on uSD card that had been in use for just under 2 years.

I can only speak of the experience that I have had, but I don't think my
usage is sufficiently atypical to not count as 'in practice' for use as
computer storage (ie I am not talking about  music players and cameras,
phones etc).

The symptom tends to be thousands of widely dispersed bad sectors appearing
almost simulataneously, in this case on a journaling filesystem, so
previously valid information goes bad without any access being made to it.
Perhaps this is wear leveling going wrong when it is trying to move less
frequently used data to parts of the flash which are very worn. Whatever it
is, it seems subjectively to be a much more rapid decline than rotating
media when it starts going wrong, and whereas rotating media usually
returns sporadic read errors when failing, I have found SSD's often
silently return the wrong data when they go bad - which I find particularly
worrying (you wouldn't want to be using a SSD in a RAID, for example).
Consequently I have started using filesystems which checksum data as well
as metadata when using flash based storage.

My usage of 2.5" SATA SSD's has not really been over a long enough period
to get a good feel for how it compares with removable flash media - I would
hope it is more robust. But when used as the only storage in a laptop
environment, I would expect much higher levels or write access than a
specially configured server. I don't pretend to know what 'typical things
people do' would be, but it isn't hard to imagine scenarios that could
result in several GB's a day for a non-technical user - downloading movies
to a laptop to watch while commuting for example... I certainly wouldn't
feel comfortable regularly rebuilding the Linux kernel on a SSD based
machine.

So from my experience, I would still tend to go along with Erik's advice
(as relayed by Steve), or perhaps be even more fastidious about backups
when using flash...



On 3 February 2018 at 20:10, Bakul Shah  wrote:

> On Sat, 03 Feb 2018 18:49:50 + "Digby R.S. Tarvin" 
> wrote:
> Digby R.S. Tarvin writes:
> >
> > My experience of running normal (read mostly) Linux filesystems on solid
> > state media is that SSD is more robust but far less reliable than
> rotating
> > media.
> >
> > MTBF for rotating media for me has been around 10 years. MTBF for SSD has
> > been about 2. And the SSD always seems to fail catastrophically -
> appearing
> > to work fine one day, then after an inexplicable crash, half the media is
> > unreadable. I assume this is something to do with the wear leveling,
> which
> > successfully gets most of the blocks to wear out at the same time with no
> > warning. If I reformat and reload the SSD to correct all the mysterious
> > corruptions, it last a few weeks, then does the same thing again.
>
> MTTF doesn't make much sense for SSDs. A 1TB SSD I bought a
> couple years ago has a rating of 300 TB written, an MTBF of 2M
> hours and a 10 year warranty. It can do over 500MB/s of
> sequential writes. If I average 9.5MB/s writes, it will last
> year. If I continuoulsy write 100MB/s, it will last under 35
> days. In contrast the life of an HDD depends on how long it
> has been spinning, seeks, temperature and load/unloads. A disk
> with 5 year warraty will likely last 5 years even if you write
> 100MB/s continuously.
>
> And consumer HDSC cards such as the ones used in cameras and
> Raspi are much much worse.
>
> In practice an SSD will last much longer than a HDD since
> average write rates are not high for the typical things people
> do.
>
>


Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-03 Thread Bakul Shah
On Sat, 03 Feb 2018 18:49:50 + "Digby R.S. Tarvin"  
wrote:
Digby R.S. Tarvin writes:
> 
> My experience of running normal (read mostly) Linux filesystems on solid
> state media is that SSD is more robust but far less reliable than rotating
> media.
> 
> MTBF for rotating media for me has been around 10 years. MTBF for SSD has
> been about 2. And the SSD always seems to fail catastrophically - appearing
> to work fine one day, then after an inexplicable crash, half the media is
> unreadable. I assume this is something to do with the wear leveling, which
> successfully gets most of the blocks to wear out at the same time with no
> warning. If I reformat and reload the SSD to correct all the mysterious
> corruptions, it last a few weeks, then does the same thing again.

MTTF doesn't make much sense for SSDs. A 1TB SSD I bought a
couple years ago has a rating of 300 TB written, an MTBF of 2M
hours and a 10 year warranty. It can do over 500MB/s of
sequential writes. If I average 9.5MB/s writes, it will last
year. If I continuoulsy write 100MB/s, it will last under 35
days. In contrast the life of an HDD depends on how long it
has been spinning, seeks, temperature and load/unloads. A disk
with 5 year warraty will likely last 5 years even if you write
100MB/s continuously.

And consumer HDSC cards such as the ones used in cameras and
Raspi are much much worse.

In practice an SSD will last much longer than a HDD since
average write rates are not high for the typical things people
do.



Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-03 Thread Digby R.S. Tarvin
My experience of running normal (read mostly) Linux filesystems on solid
state media is that SSD is more robust but far less reliable than rotating
media.

MTBF for rotating media for me has been around 10 years. MTBF for SSD has
been about 2. And the SSD always seems to fail catastrophically - appearing
to work fine one day, then after an inexplicable crash, half the media is
unreadable. I assume this is something to do with the wear leveling, which
successfully gets most of the blocks to wear out at the same time with no
warning. If I reformat and reload the SSD to correct all the mysterious
corruptions, it last a few weeks, then does the same thing again.

I have had servers running of solid state media continuously since about
2003. Using PATA to CF adapters initially, currently uSD in raspberry pi
etc, and 2.5" SATA SSD drives. I used to use mostly SCSI rotating media, so
my reliability may have been better than cheaper PC drives. I had quite a
few (probably 90%) of the 1GB Wren 7 drives retired after 10-15 years of
running 24/7 under very heavy load (in an on-air broadcast system) with no
signs of trouble. The 2.5" SATA form factor SSD's seem to last better -
perhaps indicating that the advertised capacity is a smaller proportion of
overall capacity available to level the wear over..

I don't have a large number of servers, so not really a big enough sample
to draw definite conclusions from, but enough to make me wary of relying
too much on SSD's as a panacea.

My current server configuration is a uSD system drive, with a much larger
rotating disk that spins down when not in use (generally only gets used
when a user is logged in), and an up to date backup of everything on the
uSD is kept on the rotating media. I am not keen on having SSD as a swap
device, unless you have multiple SSD's, in which case you just treat the
swap/tmp media as disposable. If I am short of ram (like on a raspberry
pi), I would prefer to have an external ram drive for swapping.

I have had the rotating media fail once in this configuration - quite
recently. 1TB 5.25", so quite a few years old.  It went through a couple of
months of taking too long to spin up when accessed after a spin down,
requiring a manual unmount to get the system to recognize it again. Then
eventually wouldn't spin up at all. The interesting thing (for me) was that
the SMART data from the drive gave it an all clear right to the end. But
unlike the SSDs, there was plenty of behavioural warning to remind me to
have the backups up to date and a spare at the ready...

So bottom line, in my experience, SSDs are great for read access time, low
power, low noise, and robustness. But they are not good for for
reliability, capacity or usage which is not read-mostly. (and RAID usage is
no substitute for backups - but that is another story)

DigbyT.

On 3 February 2018 at 16:53, hiro <23h...@gmail.com> wrote:

> not so sure about mtbf. but it's too early to tell.
>
>


Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-03 Thread hiro
not so sure about mtbf. but it's too early to tell.



Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-03 Thread Steve Simon
More the lidea of reading from the ssd first.
The way fs(4) driver works is you order your drives
with the first written at one end of the queue and first read at the other.

I assume the read rate and rotational latency of an ssd should
be better than hdd so I get the best performace.

I should add that Erik once warned me (and I believe him) that ssds are
really no more reliable in terms of MTBF than hhds, its just that they are more
robust to impact - just in case anyone thinks they last forever...

-Steve




Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-03 Thread hiro
what's the rationale behind writing to the hdd first?



Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-03 Thread Steve Simon
i would say ssd is an excellent chiice for venti, the argument is less clear 
for fossil which us much more like a traditional filesystem.

fossil and venti do not have the performance if a modern filesystem bur an ssd 
can make them fast enough for most use (i dont stream movies from my plan9 
system).

personally i have three disks mirrored, two rotating disks and, last year i 
added an sdd, they are irdered in the mirror so the ssd is read first but the 
winchesters are written first.

-Steve

> On 3 Feb 2018, at 09:39, lchg  wrote:
> 
> As I know,  fossil/venti file system is log-structured,  so it may be good 
> for flash devices, especially in extending life of flash devices.


Re: [9fans] Is fossil/venti file system a good choice for SSD?

2018-02-03 Thread Ethan Grammatikidis
On Sat, Feb 3, 2018, at 9:39 AM, lchg wrote:
> As I know,  fossil/venti file system is log-structured,  so it may be
> good for flash devices, especially in extending life of flash devices.
As far as I know the device itself will even the wear by remapping the
blocks it presents to the computer. I remember documentation for Linux
jffs saying there's no point using it on removable flash storage because
it does that, it's only raw flash memory ICs which need filesystem
support. I'm sure SSDs do their own wear leveling too, because, like
removable flash storage, they're sold for use with fat32, ntfs, etc.
--
The lyf so short, the craft so long to lerne. -- Chaucer