Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Dale]

Frank Steinmetzger wrote:
> Am Sun, Apr 16, 2023 at 05:26:15PM -0500 schrieb Dale:
>
 I'm wanting to be able to boot something from the hard drive in the
 event the OS itself won't boot.  The other day I had to dig around and
 find a bootable USB stick and also found a DVD.  Ended up with the DVD
 working best.  I already have memtest on /boot.  Thing is, I very rarely
 use it.  ;-)
>>> So in the scenario you are suggesting, is grub working, giving you a
>>> boot choice screen, and your new Gentoo install is not working so
>>> you want to choose Knoppix to repair whatever is wrong with 
>>> Gentoo? 
>> Given I have a 500GB drive, I got plenty of space.  Heck, a 10GB
>> partition each is more than enough for either Knoppix or LiveGUI.  I
>> could even store info on there about drive partitions and scripts that I
>> use a lot.  Jeez, that's a idea. 
> Back in the day, I was annoyed that whenever I needed $LIVE_SYSTEM, I had to 
> reformat an entire USB stick for that. In times when you don’t even get 
> sticks below 8 GB anymore, I found it a waste of material and useful storage 
> space.
>
> And then I found ventoy: https://www.ventoy.net/
>
> It is a mini-Bootloader which you install once to a USB device, kind-of a 
> live system of its own. But when booting it, it dynamically scans the 
> content of its device and creates a new boot menu from it. So you can put 
> many ISOs on one device as simple files, delete them, upgrade them, 
> whatever, and then you can select one to boot from. Plus, the rest of the 
> stick remains usable as storage, unlike sticks that were dd’ed with an ISO.
>
> -- Grüße | Greetings | Salut | Qapla’ Please do not share anything
> from, with or about me on any social network. The four elements:
> earth, air and firewater.

I made a USB stick with Ventoy on it but hadn't had a chance to boot
anything with it until a few days ago. I currently have the following on
ONE USB stick.


CUSTOMRESCUECD-x86_64-0.12.8.iso
KNOPPIX_V9.1DVD-2021-01-25-EN.iso
livegui-amd64-20230402T170151Z.iso
memtest86.iso
systemrescuecd-x86-5.3.2.iso


I'm having trouble with the Custom Rescue one but it could be bad since
all the others work.  CRC does try to boot but then fails.  If one were
to buy a 64GB stick, one could put a LOT of images on there.  Mine is
just a 32GB and look what I got on there. 

Anyone who boots using USB on occasion should really try this thing
out.  It is as easy as Frank says.  You install the Ventoy thing on it
and then just drop the files on there.  After that, it just works.  This
thing is awesome.  Whoever came up with this thing had a really good
slide rule. This one stick replaces five doing it the old way. 

Seriously, try this thing.  Thanks to Frank for posting about it.  I'd
never heard of it. 

Dale

:-)  :-) 

P. S. I put my old Gigabyte 770T mobo back together.  My old video card
died so I had to order one of those.  It also needed a new button
battery.  Now I'm waiting on a pack of hard drives to come in.  I
ordered a pack of five 320GB drives.  For a OS, plenty big enough.  I
also ordered a pair of USB keyboards since all I have laying around is
PS/2 types now.  On my main rig, I put in another 18GB hard drive.  I'm
replacing a 16TB on one VG and plan to use the 16TB on another VG to
replace a 8GB.  That will increase the size of both VGs.  Hopefully next
month I can order the case for the new rig.  Then I get to save up for
CPU, mobo and memory.  Just a little update for those following my puter
escapades.  LOL 

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Mon, Apr 17, 2023 at 10:45:46AM -0700 schrieb Mark Knecht:

> And I don't know that formatting ext4 or some other FS to 16K
> really helps the write amplification issue but it makes sense to
> me to match the file system blocks to the underlying flash
> block size.

The problem is finding out the write block size. This 7-year-old post says 
it’s reached 16 K: https://superuser.com/questions/976257/page-sizes-ssd

So I would say don’t bother. If everything is trimmed, there is no 
amplification. And if the disk becomes full and you get WA when writing 
itsy-bitsy 4 K files, you probably still won’t notice much difference, as 
random 4 K writes are slow anyways and how often do you write thousands of
4 K files outside of portage?

Erase block sizes probably go into the megabytes these days:
https://news.ycombinator.com/item?id=29165202

Some more detailed explanation:
https://spdk.io/doc/ssd_internals.html
  “For each erase block, each bit may be written to (i.e. have its bit 
  flipped from 0 to 1) with bit-granularity once. In order to write to the 
  erase block a second time, the entire block must be erased (i.e. all bits 
  in the block are flipped back to 0).”

This sounds like my initial statement was partially wrong – trimming does 
cause writing zeroes, because that’s what an erase does. But it still 
prevents write amplification (and one extra erase cycle) because 
neighbouring blocks don’t need to be read and written back.

> Real speed testing would be required to ensure reading
> 16K blocks doesn't slow him down though.

Here are some numbers and a conclusion gathered from a read test:
https://superuser.com/questions/728858/how-to-determine-ssds-nand-erase-block-size

Unless I positively need the speed for high-performance computing, I’d 
rather keep the smaller granularity for more capacity at low file sizes.

A problem is what some call “parts lottery” these days: manufacturers 
promise some performance on the data sheet (“up to xxx”), but not with which 
parts they want to achieve this (types of flash chips, TLC/QLC, controller, 
DRAM and so on). Meaning during the lifetime of a product, its internals may 
change and as a consequence those specs are not in the data sheet:

https://unix.stackexchange.com/questions/334804/is-there-a-way-to-find-out-ssd-page-size-on-linux-unix-what-is-physical-block
  “There is no standard way for a SSD to report its page size or erase block 
  size. Few if any manufacturers report them in the datasheets. (Because 
  they may change during the lifetime of a SKU, for example because of 
  changing suppliers.)
  For practical use just align all your data structures (partitions, payload 
  of LUKS containers, LVM logical volumes) to 1 or 2 MiB boundaries. It's an 
  SSD after all--it is designed to cope with usual filesystems, such as NTFS 
  (which uses 4 KiB allocation units).”

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

The worst disease is indifference. So what?


signature.asc
Description: PGP signature

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Dale]

Mark Knecht wrote:
>
>
> On Mon, Apr 17, 2023 at 2:41 AM Wols Lists  > wrote:
> >
> > On 17/04/2023 02:14, Dale wrote:
> > > My current install is over a decade old.  My /boot partition is about
> > > 375MBs.  I should have made it larger but at the time, I booted CD/DVD
> > > media when needed.  I didn't have USB sticks at the time.  This
> time, I
> > > plan to make some changes.  If I put Knoppix and/or Gentoo LiveGUI in
> > > /boot, it will be larger.  Much larger.  Mark's idea is best tho. 
> If I
> > > can get Grub to work and boot it.
> >
> > If you dd your boot partition across, you can copy it into a larger
> > partition on the new drive, and then just expand the filesystem.
> >
> > So changing partition sizes isn't a problem if you want to just copy
> > your system drive onto a new disk.
> >
> > Cheers,
> > Wol
>
> I'm not sure I'd use dd in this case. If he's moving from an HDD with
> a 4K block size and a 4K file system block size to an SDD with a 16K 
> physical block size he might want to consider changing the filesystem 
> block size to 16K which should help on the write amplification side.
>
> Maybe dd can do that but I wouldn't think so.
>
> And I don't know that formatting ext4 or some other FS to 16K 
> really helps the write amplification issue but it makes sense to
> me to match the file system blocks to the underlying flash
> block size. Real speed testing would be required to ensure reading
> 16K blocks doesn't slow him down though.
>
> Just a thought,
> Mark


I still haven't got around to partitioning the drive or anything so I'm
glad you mentioned the block size.  I need to try and remember that.  It
may detect it itself but may not.  I'd rather fix it now than wish I did
later on.  I assume that setting is in the man page. 

Thanks for that tidbit.  Now to remember it.  :/

Dale

:-)  :-) 

P. s.  It's garden time for folks around here.  I been busy the past few
days.  Tractor and tiller too.

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Mark Knecht]

On Mon, Apr 17, 2023 at 2:41 AM Wols Lists  wrote:
>
> On 17/04/2023 02:14, Dale wrote:
> > My current install is over a decade old.  My /boot partition is about
> > 375MBs.  I should have made it larger but at the time, I booted CD/DVD
> > media when needed.  I didn't have USB sticks at the time.  This time, I
> > plan to make some changes.  If I put Knoppix and/or Gentoo LiveGUI in
> > /boot, it will be larger.  Much larger.  Mark's idea is best tho.  If I
> > can get Grub to work and boot it.
>
> If you dd your boot partition across, you can copy it into a larger
> partition on the new drive, and then just expand the filesystem.
>
> So changing partition sizes isn't a problem if you want to just copy
> your system drive onto a new disk.
>
> Cheers,
> Wol

I'm not sure I'd use dd in this case. If he's moving from an HDD with
a 4K block size and a 4K file system block size to an SDD with a 16K
physical block size he might want to consider changing the filesystem
block size to 16K which should help on the write amplification side.

Maybe dd can do that but I wouldn't think so.

And I don't know that formatting ext4 or some other FS to 16K
really helps the write amplification issue but it makes sense to
me to match the file system blocks to the underlying flash
block size. Real speed testing would be required to ensure reading
16K blocks doesn't slow him down though.

Just a thought,
Mark

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Wols Lists]

On 17/04/2023 02:14, Dale wrote:

My current install is over a decade old.  My /boot partition is about
375MBs.  I should have made it larger but at the time, I booted CD/DVD
media when needed.  I didn't have USB sticks at the time.  This time, I
plan to make some changes.  If I put Knoppix and/or Gentoo LiveGUI in
/boot, it will be larger.  Much larger.  Mark's idea is best tho.  If I
can get Grub to work and boot it.


If you dd your boot partition across, you can copy it into a larger 
partition on the new drive, and then just expand the filesystem.


So changing partition sizes isn't a problem if you want to just copy 
your system drive onto a new disk.


Cheers,
Wol

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Dale]

Frank Steinmetzger wrote:
> Am Sun, Apr 16, 2023 at 05:26:15PM -0500 schrieb Dale:
>
 I'm wanting to be able to boot something from the hard drive in the
 event the OS itself won't boot.  The other day I had to dig around and
 find a bootable USB stick and also found a DVD.  Ended up with the DVD
 working best.  I already have memtest on /boot.  Thing is, I very rarely
 use it.  ;-)
>>> So in the scenario you are suggesting, is grub working, giving you a
>>> boot choice screen, and your new Gentoo install is not working so
>>> you want to choose Knoppix to repair whatever is wrong with 
>>> Gentoo? 
>> Given I have a 500GB drive, I got plenty of space.  Heck, a 10GB
>> partition each is more than enough for either Knoppix or LiveGUI.  I
>> could even store info on there about drive partitions and scripts that I
>> use a lot.  Jeez, that's a idea. 
> Back in the day, I was annoyed that whenever I needed $LIVE_SYSTEM, I had to 
> reformat an entire USB stick for that. In times when you don’t even get 
> sticks below 8 GB anymore, I found it a waste of material and useful storage 
> space.
>
> And then I found ventoy: https://www.ventoy.net/
>
> It is a mini-Bootloader which you install once to a USB device, kind-of a 
> live system of its own. But when booting it, it dynamically scans the 
> content of its device and creates a new boot menu from it. So you can put 
> many ISOs on one device as simple files, delete them, upgrade them, 
> whatever, and then you can select one to boot from. Plus, the rest of the 
> stick remains usable as storage, unlike sticks that were dd’ed with an ISO.
>

My current install is over a decade old.  My /boot partition is about
375MBs.  I should have made it larger but at the time, I booted CD/DVD
media when needed.  I didn't have USB sticks at the time.  This time, I
plan to make some changes.  If I put Knoppix and/or Gentoo LiveGUI in
/boot, it will be larger.  Much larger.  Mark's idea is best tho.  If I
can get Grub to work and boot it. 

I'll look into your link more.  It sounds interesting but can't figure
out exactly how it works.  May check youtube for a video.  Should clear
up the muddy water. 

Dale

:-)  :-) 

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Mark Knecht]

On Sun, Apr 16, 2023 at 3:18 PM Frank Steinmetzger  wrote:
>
> Am Sun, Apr 16, 2023 at 01:22:32PM -0700 schrieb Mark Knecht:
> > Frank,
> >Thank you for the in-depth explanation.
> >
> >I need to do some study before commenting further other than to say
> > so far I'm finding different comments depending on whether it's
> > an SSD or an M.2 drive.
>
> Uhm, I think you mix up some terms here. An M.2 drive *is* an SSD
> (literally, as the name says, a solid state drive). By “SSD”, did you mean
> the classic laptop form factor for SATA HDDs and SSDs?
>
> Because M.2 is also only a physical form factor. It supports both NVMe and
> SATA. While NVMe is more modern and better suited for solid state media
and
> their properties, in the end it is still only a data protocol to transfer
> data to and fro.
>

No, I don't believe I've mixed them up but if you see something I'm wrong
about
let me know.

When I speak of SSDs I do mean devices that are marketed as SSDs &
probably use SATA.

When I speak of M.2 I mean what you and I both call M.2.

While SSD & M.2 are both Flash devices they don't  provide the same info
when queried by smartctl which makes a direct comparison more
difficult.

Depending on the manufacturer & the foundry they build the chips in the
technologies in these devices can be quite different independent of whether
they are M.2 or SSD.

1) Dale's Samsung 870 EVO - V-NAND TLC (8-bits/cell) and 600TB written
2) My Crucial 1TB M.2 is QLC (16 bits/cell) and 450TB written
3) My Sabrent 1TB M.2 is TLC (8 bits/cell) and 700TB written
4) My Crucial 250GB is unknown because Crucial sells 5 versions
that come from different fabs and have different specs.

All 4 drives are warranted for 5 years or hitting the TB written value.

All 4 drives have 16K page sizes.

That said, I've been using the Crucial on my Kubuntu dual boot for over a
year and only have 28TB written so I'm a long way from the 450TB spec and
likely won't come close in 5 years. (If I'm even still using this machine.)

On the Windows side which I use far less I've only written about 2TB total.

In my case the workloads are generally fairly large files. They are
generally
either 24MB photo files for astrophotography or audio recording files which
are typically 50-100K. Neither of them are 'modified' and need to be
rewritten. They are either saved or deleted.

Whether the write amplification makes a difference or not in real life
I don't know. I'm sure for some work loads it does but the 'percent
used' value that smartctl returns is 2% for the Crucial and 0% for
the Sabrent so both appear to have a lot of life left in them.

Mark

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Sun, Apr 16, 2023 at 05:26:15PM -0500 schrieb Dale:

> > > I'm wanting to be able to boot something from the hard drive in the
> > > event the OS itself won't boot.  The other day I had to dig around and
> > > find a bootable USB stick and also found a DVD.  Ended up with the DVD
> > > working best.  I already have memtest on /boot.  Thing is, I very rarely
> > > use it.  ;-)
> >
> > So in the scenario you are suggesting, is grub working, giving you a
> > boot choice screen, and your new Gentoo install is not working so
> > you want to choose Knoppix to repair whatever is wrong with 
> > Gentoo? 
> 
> Given I have a 500GB drive, I got plenty of space.  Heck, a 10GB
> partition each is more than enough for either Knoppix or LiveGUI.  I
> could even store info on there about drive partitions and scripts that I
> use a lot.  Jeez, that's a idea. 

Back in the day, I was annoyed that whenever I needed $LIVE_SYSTEM, I had to 
reformat an entire USB stick for that. In times when you don’t even get 
sticks below 8 GB anymore, I found it a waste of material and useful storage 
space.

And then I found ventoy: https://www.ventoy.net/

It is a mini-Bootloader which you install once to a USB device, kind-of a 
live system of its own. But when booting it, it dynamically scans the 
content of its device and creates a new boot menu from it. So you can put 
many ISOs on one device as simple files, delete them, upgrade them, 
whatever, and then you can select one to boot from. Plus, the rest of the 
stick remains usable as storage, unlike sticks that were dd’ed with an ISO.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The four elements: earth, air and firewater.


signature.asc
Description: PGP signature

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Dale]

Mark Knecht wrote:
>
>
> On Sun, Apr 16, 2023 at 11:54 AM Dale  > wrote:
> >
> > Mark Knecht wrote:
> > > <<< SNIP >>>
> > > I guess I don't understand why you would put Knoppix in the boot
> > > partition vs somewhere else. Is this for some sort of recovery
> > > process you're comfortable with vs recovering from a bootable DVD?
> > >
> > > - Mark
> >
> > I'm wanting to be able to boot something from the hard drive in the
> > event the OS itself won't boot.  The other day I had to dig around and
> > find a bootable USB stick and also found a DVD.  Ended up with the DVD
> > working best.  I already have memtest on /boot.  Thing is, I very rarely
> > use it.  ;-)
>
> So in the scenario you are suggesting, is grub working, giving you a
> boot choice screen, and your new Gentoo install is not working so
> you want to choose Knoppix to repair whatever is wrong with 
> Gentoo? 
>
> If that's the case why shoehorn Knoppix into the boot partition
> vs just give it its own partition? 
>
>


Dang Mark, I hadn't thought of that.    See, that's why
you need to stick around here.  ROFL  I wonder, can I do that with the
Gentoo LiveGUI too??  Anyone know?  I may need help getting Grub to see
those tho.  I let it do it's thing with my kernels but I have no idea on
pointing it to something else. 

Given I have a 500GB drive, I got plenty of space.  Heck, a 10GB
partition each is more than enough for either Knoppix or LiveGUI.  I
could even store info on there about drive partitions and scripts that I
use a lot.  Jeez, that's a idea. 

Thanks.

Dale

:-)  :-)  :-)  :-) 

P. S. Extra happy there.  ;-)

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Sun, Apr 16, 2023 at 01:22:32PM -0700 schrieb Mark Knecht:
> Frank,
>Thank you for the in-depth explanation.
> 
>I need to do some study before commenting further other than to say
> so far I'm finding different comments depending on whether it's
> an SSD or an M.2 drive.

Uhm, I think you mix up some terms here. An M.2 drive *is* an SSD 
(literally, as the name says, a solid state drive). By “SSD”, did you mean 
the classic laptop form factor for SATA HDDs and SSDs?

Because M.2 is also only a physical form factor. It supports both NVMe and 
SATA. While NVMe is more modern and better suited for solid state media and 
their properties, in the end it is still only a data protocol to transfer 
data to and fro.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

„He who prefers security to freedom deserves to be a slave.“ – Aristotle


signature.asc
Description: PGP signature

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Mark Knecht]

Frank,
   Thank you for the in-depth explanation.

   I need to do some study before commenting further other than to say
so far I'm finding different comments depending on whether it's
an SSD or an M.2 drive.

Much appreciated,
Mark

On Sun, Apr 16, 2023 at 11:08 AM Frank Steinmetzger  wrote:

> Am Sun, Apr 16, 2023 at 08:08:59AM -0700 schrieb Mark Knecht:
>
> > If you have an SSD or nvme drive installed then fstrim should be
> > installed and run on a regular basis. However it's not 'required'.
> >
> > Your system will still work, but after all blocks on the drive have
> > been used for file storage and later deleted, if they are not
> > written back to zeros then the next time you go to use that
> > block the write will be slower as the write must first write
> > zeros and then your data.
> >
> > fstrim does the write to zeros so that during normal operation
> > you don't wait.
>
> That is not quite correct. Trimming is about the oppisite of what you say,
> namely to *not* rewrite areas. Flash memory can only be written to in
> relatively large blocks. So if your file system wants to write 4 KiB, the
> drive needs to read all the many kB around it (several hundreds at least,
> perhaps eben MiBs, I’m not certain), change the small part in question and
> write the whole block back. This is called write amplification. This also
> occurs on hard drives, for example when you run a database which uses 4
> kiB
> datafile chunks, but on a file system with larger sectors. Then the file
> system is the cause for write amplification.
>
> If the SSD knew beforehand that the area is unused, it does not need to
> read
> it all in and then write it back. The SSD controller has no knowledge of
> file systems. And this is where trim comes in: it does know file systems,
> detects the unused areas and translates that info for the drive
> controller.
> Also, only trimmed areas (i.e. areas the controller knows are unused) can
> be
> used for wear leveling.
>
> I even think that If you read from a trimmed area, the controller does not
> actually read the flash device, but simply returns zeroes. This is
> basically
> what a quick erase does; it trims the entire drive, which takes only a few
> seconds, and then all the data has become inaccessible (unless you address
> the memory chips directly). It is similar to deleting a file: you erase
> its
> entry in the directory, but not the actual payload bytes.
>
> AFAIK, SMR HDDs also support trim these days, so they don’t need to do
> their
> SMR reshuffling. I have a WD Passport Ultra external 2.5″ HDD with 5 TB,
> and
> it supports trim. However, a WD Elements 2.5″ 4 TB does not. Perhaps
> because
> it is a cheaper series. Every laptop HDD of 2 (or even 1) TB is SMR.
>
> --
> Grüße | Greetings | Salut | Qapla’
> Please do not share anything from, with or about me on any social network.
>
> “It is hard to be a conquering hero when it is not in your nature.”
>   – Captain Hans Geering, ’Allo ’Allo
>

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Mark Knecht]

On Sun, Apr 16, 2023 at 11:54 AM Dale  wrote:
>
> Mark Knecht wrote:
> > <<< SNIP >>>
> > I guess I don't understand why you would put Knoppix in the boot
> > partition vs somewhere else. Is this for some sort of recovery
> > process you're comfortable with vs recovering from a bootable DVD?
> >
> > - Mark
>
> I'm wanting to be able to boot something from the hard drive in the
> event the OS itself won't boot.  The other day I had to dig around and
> find a bootable USB stick and also found a DVD.  Ended up with the DVD
> working best.  I already have memtest on /boot.  Thing is, I very rarely
> use it.  ;-)

So in the scenario you are suggesting, is grub working, giving you a
boot choice screen, and your new Gentoo install is not working so
you want to choose Knoppix to repair whatever is wrong with
Gentoo?

If that's the case why shoehorn Knoppix into the boot partition
vs just give it its own partition?

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Dale]

Mark Knecht wrote:
> <<< SNIP >>>
> I guess I don't understand why you would put Knoppix in the boot 
> partition vs somewhere else. Is this for some sort of recovery 
> process you're comfortable with vs recovering from a bootable DVD?
>
> - Mark

I'm wanting to be able to boot something from the hard drive in the
event the OS itself won't boot.  The other day I had to dig around and
find a bootable USB stick and also found a DVD.  Ended up with the DVD
working best.  I already have memtest on /boot.  Thing is, I very rarely
use it.  ;-)

Dale

:-)  :-) 

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Mark Knecht]

On Sun, Apr 16, 2023 at 9:55 AM Dale  wrote:

>
> I think what I read is that it is done automatically.  They could have
> meant a cron job.  I don't think they said how, just that it is already
> set up to do it.  Firmware was mentioned in the thread somewhere so I
> thought maybe that was it.  Either way, fstrim is installed here.  It's
> part of util-linux and that is pulled in by several packages.  I doubt
> it will be going away here anytime soon given the long list of packages
> that need it. Just to set up a cron job for it.  Remembering the steps
> for that will take time tho.  o_O
>

Hey - it's the Internet. I thought we could trust everything we read...


>
>
> I've thought of a few options myself.  I sort of have a OS copy/backup
> already.  I currently do the compiling in a chroot on a separate drive.
> I then copy the compiled packages and use the -k option to update the
> live OS.  I'll continue to do that when I start booting from the SSD.
> That should limit writes and such to the SSD.  I also got to rearrange
> things so I can put swap on that spare drive I compile on.  I don't want
> swap on a SSD.  I wish this thing would stop using swap completely.  I
> have swappiness set to 1 already and it still uses swap.
>

Well, that sounds like a solution to do your emerge work although
you're limited to the speed of that hard drive. If it's done in the
background then maybe you don't care.

> Right now, I'm debating the size of /boot.  Knoppix is pretty large.
> The Gentoo LiveGUI thingy is too.  So, it will have to be larger than
> the few hundred megabytes my current one is.  I'm thinking 10GBs or so.
> Maybe 12GBs to make sure I'm good to go for the foreseeable future.
> They may limit them to DVD size right now but one day they could pass
> that limit by.  Software isn't getting smaller.  Besides, USB is the
> thing now.
>

I guess I don't understand why you would put Knoppix in the boot
partition vs somewhere else. Is this for some sort of recovery
process you're comfortable with vs recovering from a bootable DVD?

- Mark

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Sun, Apr 16, 2023 at 08:08:59AM -0700 schrieb Mark Knecht:

> If you have an SSD or nvme drive installed then fstrim should be
> installed and run on a regular basis. However it's not 'required'.
> 
> Your system will still work, but after all blocks on the drive have
> been used for file storage and later deleted, if they are not
> written back to zeros then the next time you go to use that
> block the write will be slower as the write must first write
> zeros and then your data.
> 
> fstrim does the write to zeros so that during normal operation
> you don't wait.

That is not quite correct. Trimming is about the oppisite of what you say, 
namely to *not* rewrite areas. Flash memory can only be written to in 
relatively large blocks. So if your file system wants to write 4 KiB, the 
drive needs to read all the many kB around it (several hundreds at least, 
perhaps eben MiBs, I’m not certain), change the small part in question and 
write the whole block back. This is called write amplification. This also 
occurs on hard drives, for example when you run a database which uses 4 kiB 
datafile chunks, but on a file system with larger sectors. Then the file 
system is the cause for write amplification.

If the SSD knew beforehand that the area is unused, it does not need to read 
it all in and then write it back. The SSD controller has no knowledge of 
file systems. And this is where trim comes in: it does know file systems, 
detects the unused areas and translates that info for the drive controller. 
Also, only trimmed areas (i.e. areas the controller knows are unused) can be 
used for wear leveling.

I even think that If you read from a trimmed area, the controller does not 
actually read the flash device, but simply returns zeroes. This is basically 
what a quick erase does; it trims the entire drive, which takes only a few 
seconds, and then all the data has become inaccessible (unless you address 
the memory chips directly). It is similar to deleting a file: you erase its 
entry in the directory, but not the actual payload bytes.

AFAIK, SMR HDDs also support trim these days, so they don’t need to do their 
SMR reshuffling. I have a WD Passport Ultra external 2.5″ HDD with 5 TB, and 
it supports trim. However, a WD Elements 2.5″ 4 TB does not. Perhaps because 
it is a cheaper series. Every laptop HDD of 2 (or even 1) TB is SMR.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

“It is hard to be a conquering hero when it is not in your nature.”
  – Captain Hans Geering, ’Allo ’Allo


signature.asc
Description: PGP signature

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Jorge Almeida]

On Sun, Apr 16, 2023 at 4:09 PM Mark Knecht  wrote:

> Sorry for any confusion. Time to unsubscribe from this list
> I guess and leave you all to your beloved distro.
>
Please don't.
I doubt someone is pissed off with you.

Jorge Almeida

> Bye,
See above.
> Mark

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Dale]

Mark Knecht wrote:
> 
> >
> >
> > Oh, please, don't go anywhere.    We already lost the long
> term Alan.  BTW, I checked on him a while back.  He's still OK.  It's
> been a while tho.
> >
>
> Dale - I touched - truly. I'm happy to stick around but I no longer 
> run Gentoo and don't want to cause problems or piss off the 
> folks who need to be here. 
>
> > I read during a google search that some distros handle this sort of
> thing automatically, some sort of firmware thing or something.  I
> figured Gentoo didn't, it rarely does since that is the point of
> Gentoo.  So, no harm.  Heck, I just now applied power to the thing.  I
> don't even have it partitioned or anything yet.  Just rebooted after
> rearranging all the cables, adding power splitter etc etc.
>
> I don't know of any distros that do any of this in firmware. Maybe someone
> else can address that. Many distros now install fstrim by default and 
> update crontab. The Ubuntu family does, which I didn't know before this
> thread. However it is on both my Kubuntu machine and my Ubuntu
> Server machine.
>

I think what I read is that it is done automatically.  They could have
meant a cron job.  I don't think they said how, just that it is already
set up to do it.  Firmware was mentioned in the thread somewhere so I
thought maybe that was it.  Either way, fstrim is installed here.  It's
part of util-linux and that is pulled in by several packages.  I doubt
it will be going away here anytime soon given the long list of packages
that need it. Just to set up a cron job for it.  Remembering the steps
for that will take time tho.  o_O


> BTW - Windows does this in Disk Defragmenter but you have to 
> schedule it yourself according to Bard and ChatGPT. I'll be in 
> Windows later today to record some new music and plan to look into
> that then.
>
> Answering the question from below - weekly is what was set up by 
> default here. If your drive isn't near full and you're not writing a
> lot of new
> data on it each week then weekly would seem reasonable to me. 
> However being that you are running Gentoo and hence compiling
> lots and lots and lots of stuff every week it's possible that you 
> might _possibly_ want to run fstrim more often if your intermediate 
> files are going to this drive. 
>
>
> >
> > I do have one gripe.  Why can't drive makers pick a screw size and
> stick to it on ALL drives?  It took some digging to find a screw that
> would fit.  Some I bought that are supposed to work on SSDs were to
> short.  It would likely work on a metal adapter but not a thicker
> plastic one.  Luckily, I found 4 screws.  No clue where they came
> from.  Just in my junk box.  Before this week, never laid eyes on a
> SSD before.  Anyone know the thread size and count on those things?  I
> want to order a few, just in case.
>
> I second your gripe. I've purchased a couple of PC builder screw 
> sets from Amazon.
>  
> >
> > Is running fstrim once a week to often?  I update my OS once a week
> but given the amount of extra space, I'd think once a month would be
> often enough.  After all, it is 500GB and I'll likely only use less
> than half of that.  Most of the extra space will be extra boot options
> like Knoppix or something.  I'm just thinking it would give it a
> longer life.  Maybe my thinking is wrong???
> >
> > Now to play with this thing.  I got to remember what all has to be
> copied over so I can boot the new thing.  :/  Been ages since I moved
> a OS to another hard drive.  Maybe a reinstall would work better.  :-\
> >
>
> I think you have at least 3 options to play with the drive:
>
> 1) It's Gentoo so install from scratch. You'll feel great 
> if it works. It will only take you a day or two.
>
> 2) Possibly dd the old drive to the SSD. If the new 
> SSD boots as the same /dev/sdX device it should 
> work, maybe, maybe not.
>
> 3) If you have another SATA port then dual boot, 
> either with Gentoo on both or something simple 
> like Kubuntu. A base Kubuntu install takes about
> 15 minutes and will probably give you its own
> dual boot grub config. When you're sick of Kubuntu
> you can once again install Gentoo.
>
> Good luck no matter what path you take.
>
> Mark


I've thought of a few options myself.  I sort of have a OS copy/backup
already.  I currently do the compiling in a chroot on a separate drive. 
I then copy the compiled packages and use the -k option to update the
live OS.  I'll continue to do that when I start booting from the SSD. 
That should limit writes and such to the SSD.  I also got to rearrange
things so I can put swap on that spare drive I compile on.  I don't want
swap on a SSD.  I wish this thing would stop using swap completely.  I
have swappiness set to 1 already and it still uses swap.

Right now, I'm debating the size of /boot.  Knoppix is pretty large. 
The Gentoo LiveGUI thingy is too.  So, it will have to be larger than
the few hundred megabytes my current one is.  I'm thinking 10GBs or so. 
Maybe 12GBs to make sure 

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Mark Knecht]

>
>
> Oh, please, don't go anywhere.We already lost the long term
Alan.  BTW, I checked on him a while back.  He's still OK.  It's been a
while tho.
>

Dale - I touched - truly. I'm happy to stick around but I no longer
run Gentoo and don't want to cause problems or piss off the
folks who need to be here.

> I read during a google search that some distros handle this sort of thing
automatically, some sort of firmware thing or something.  I figured Gentoo
didn't, it rarely does since that is the point of Gentoo.  So, no harm.
Heck, I just now applied power to the thing.  I don't even have it
partitioned or anything yet.  Just rebooted after rearranging all the
cables, adding power splitter etc etc.

I don't know of any distros that do any of this in firmware. Maybe someone
else can address that. Many distros now install fstrim by default and
update crontab. The Ubuntu family does, which I didn't know before this
thread. However it is on both my Kubuntu machine and my Ubuntu
Server machine.

BTW - Windows does this in Disk Defragmenter but you have to
schedule it yourself according to Bard and ChatGPT. I'll be in
Windows later today to record some new music and plan to look into
that then.

Answering the question from below - weekly is what was set up by
default here. If your drive isn't near full and you're not writing a lot of
new
data on it each week then weekly would seem reasonable to me.
However being that you are running Gentoo and hence compiling
lots and lots and lots of stuff every week it's possible that you
might _possibly_ want to run fstrim more often if your intermediate
files are going to this drive.


>
> I do have one gripe.  Why can't drive makers pick a screw size and stick
to it on ALL drives?  It took some digging to find a screw that would fit.
Some I bought that are supposed to work on SSDs were to short.  It would
likely work on a metal adapter but not a thicker plastic one.  Luckily, I
found 4 screws.  No clue where they came from.  Just in my junk box.
Before this week, never laid eyes on a SSD before.  Anyone know the thread
size and count on those things?  I want to order a few, just in case.

I second your gripe. I've purchased a couple of PC builder screw
sets from Amazon.

>
> Is running fstrim once a week to often?  I update my OS once a week but
given the amount of extra space, I'd think once a month would be often
enough.  After all, it is 500GB and I'll likely only use less than half of
that.  Most of the extra space will be extra boot options like Knoppix or
something.  I'm just thinking it would give it a longer life.  Maybe my
thinking is wrong???
>
> Now to play with this thing.  I got to remember what all has to be copied
over so I can boot the new thing.  :/  Been ages since I moved a OS to
another hard drive.  Maybe a reinstall would work better.  :-\
>

I think you have at least 3 options to play with the drive:

1) It's Gentoo so install from scratch. You'll feel great
if it works. It will only take you a day or two.

2) Possibly dd the old drive to the SSD. If the new
SSD boots as the same /dev/sdX device it should
work, maybe, maybe not.

3) If you have another SATA port then dual boot,
either with Gentoo on both or something simple
like Kubuntu. A base Kubuntu install takes about
15 minutes and will probably give you its own
dual boot grub config. When you're sick of Kubuntu
you can once again install Gentoo.

Good luck no matter what path you take.

Mark

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Dale]

Mark Knecht wrote:
>
>
> On Sun, Apr 16, 2023 at 1:44 AM William Kenworthy  > wrote:
> >
> >
> > On 16/4/23 15:18, Peter Humphrey wrote:
> > > On Sunday, 16 April 2023 02:47:00 BST William Kenworthy wrote:
> > >
> > >> look into mount options for SSD's (discard option) and "fstrim" for
> > >> maintenance. (read up on trimmimg - doing a manual trim before
> the drive
> > >> reaches full allocation (they delete files, but do not erase them
> > >> because erasing is time consuming so its an OS controlled
> operation) or
> > >> auto trimming (which can cause serious pauses at awkward times) can
> > >> prevent serious performance degradation as it has to erase before
> > >> writing.  I am not sure of the current status but in the early
> days of
> > >> SSD's, this was serious concern.
> > > In short, see https://wiki.gentoo.org/wiki/SSD .  :)
> > >
> > Excellent, condenses it nicely.
> >
> > BillK
> >
> >
>
> OK Dale, I'm completely wrong, but also 'slightly' right.
>
> If you have an SSD or nvme drive installed then fstrim should be 
> installed and run on a regular basis. However it's not 'required'.
>
> Your system will still work, but after all blocks on the drive have
> been used for file storage and later deleted, if they are not
> written back to zeros then the next time you go to use that
> block the write will be slower as the write must first write
> zeros and then your data.
>
> fstrim does the write to zeros so that during normal operation
> you don't wait.
>
> I've become so completely used to Kubuntu that I had to read
> that this is all set up automatically when the system finds an
> SSD or nvme. In Gentoo land you have to do this yourself.
>
> Sorry for any confusion. Time to unsubscribe from this list
> I guess and leave you all to your beloved distro.
>
> Bye,
> Mark


Oh, please, don't go anywhere.    We already lost the long term
Alan.  BTW, I checked on him a while back.  He's still OK.  It's been a
while tho. 

I read during a google search that some distros handle this sort of
thing automatically, some sort of firmware thing or something.  I
figured Gentoo didn't, it rarely does since that is the point of
Gentoo.  So, no harm.  Heck, I just now applied power to the thing.  I
don't even have it partitioned or anything yet.  Just rebooted after
rearranging all the cables, adding power splitter etc etc.

I do have one gripe.  Why can't drive makers pick a screw size and stick
to it on ALL drives?  It took some digging to find a screw that would
fit.  Some I bought that are supposed to work on SSDs were to short.  It
would likely work on a metal adapter but not a thicker plastic one. 
Luckily, I found 4 screws.  No clue where they came from.  Just in my
junk box.  Before this week, never laid eyes on a SSD before.  Anyone
know the thread size and count on those things?  I want to order a few,
just in case. 

Is running fstrim once a week to often?  I update my OS once a week but
given the amount of extra space, I'd think once a month would be often
enough.  After all, it is 500GB and I'll likely only use less than half
of that.  Most of the extra space will be extra boot options like
Knoppix or something.  I'm just thinking it would give it a longer
life.  Maybe my thinking is wrong???

Now to play with this thing.  I got to remember what all has to be
copied over so I can boot the new thing.  :/  Been ages since I moved a
OS to another hard drive.  Maybe a reinstall would work better.  :-\

Thanks to all. 

Dale

:-)  :-) 

P. S.  CCing Mark just in case. 

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Mark Knecht]

On Sun, Apr 16, 2023 at 1:44 AM William Kenworthy 
wrote:
>
>
> On 16/4/23 15:18, Peter Humphrey wrote:
> > On Sunday, 16 April 2023 02:47:00 BST William Kenworthy wrote:
> >
> >> look into mount options for SSD's (discard option) and "fstrim" for
> >> maintenance. (read up on trimmimg - doing a manual trim before the
drive
> >> reaches full allocation (they delete files, but do not erase them
> >> because erasing is time consuming so its an OS controlled operation) or
> >> auto trimming (which can cause serious pauses at awkward times) can
> >> prevent serious performance degradation as it has to erase before
> >> writing.  I am not sure of the current status but in the early days of
> >> SSD's, this was serious concern.
> > In short, see https://wiki.gentoo.org/wiki/SSD .  :)
> >
> Excellent, condenses it nicely.
>
> BillK
>
>

OK Dale, I'm completely wrong, but also 'slightly' right.

If you have an SSD or nvme drive installed then fstrim should be
installed and run on a regular basis. However it's not 'required'.

Your system will still work, but after all blocks on the drive have
been used for file storage and later deleted, if they are not
written back to zeros then the next time you go to use that
block the write will be slower as the write must first write
zeros and then your data.

fstrim does the write to zeros so that during normal operation
you don't wait.

I've become so completely used to Kubuntu that I had to read
that this is all set up automatically when the system finds an
SSD or nvme. In Gentoo land you have to do this yourself.

Sorry for any confusion. Time to unsubscribe from this list
I guess and leave you all to your beloved distro.

Bye,
Mark

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[William Kenworthy]

On 16/4/23 15:18, Peter Humphrey wrote:

On Sunday, 16 April 2023 02:47:00 BST William Kenworthy wrote:


look into mount options for SSD's (discard option) and "fstrim" for
maintenance. (read up on trimmimg - doing a manual trim before the drive
reaches full allocation (they delete files, but do not erase them
because erasing is time consuming so its an OS controlled operation) or
auto trimming (which can cause serious pauses at awkward times) can
prevent serious performance degradation as it has to erase before
writing.  I am not sure of the current status but in the early days of
SSD's, this was serious concern.

In short, see https://wiki.gentoo.org/wiki/SSD .  :)


Excellent, condenses it nicely.

BillK

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Peter Humphrey]

On Sunday, 16 April 2023 02:47:00 BST William Kenworthy wrote:

> look into mount options for SSD's (discard option) and "fstrim" for
> maintenance. (read up on trimmimg - doing a manual trim before the drive
> reaches full allocation (they delete files, but do not erase them
> because erasing is time consuming so its an OS controlled operation) or
> auto trimming (which can cause serious pauses at awkward times) can
> prevent serious performance degradation as it has to erase before
> writing.  I am not sure of the current status but in the early days of
> SSD's, this was serious concern.

In short, see https://wiki.gentoo.org/wiki/SSD .  :)

-- 
Regards,
Peter.

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[William Kenworthy]

On 16/4/23 06:47, Dale wrote:

Howdy,

I finally broke down and bought a SSD.  It's a Samsung V-Nand 870 EVO
500GB.  My current OS sits on a 160GB drive so should be plenty.  I plan
to even add a boot image for the Gentoo LiveGUI thingy, maybe Knoppix or
something plus my usual OS.  By the way, caught one for sale for
$40.00.  It has a production date of 5/2021.

My question is this.  Do I need anything special in the kernel or
special fstab options for this thing?  I know at one point there was
folks having problems with certain settings.  I did some googling and it
seems to be worked out but I want to be sure I don't blow this thing up
or something.

Anything else that makes these special?  Any tips or tricks?

Dale

:-)  :-)

P. S.  I'm hoping this will make my system a little more responsive.
Maybe.  Either way, that 160GB drive is getting a little full.



look into mount options for SSD's (discard option) and "fstrim" for 
maintenance. (read up on trimmimg - doing a manual trim before the drive 
reaches full allocation (they delete files, but do not erase them 
because erasing is time consuming so its an OS controlled operation) or 
auto trimming (which can cause serious pauses at awkward times) can 
prevent serious performance degradation as it has to erase before 
writing.  I am not sure of the current status but in the early days of 
SSD's, this was serious concern.


BillK


BillK

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[thelma]

On 4/15/23 17:24, Mark Knecht wrote:



On Sat, Apr 15, 2023 at 3:47 PM Dale mailto:rdalek1...@gmail.com>> wrote:
 >
 > Howdy,
 >
 > I finally broke down and bought a SSD.  It's a Samsung V-Nand 870 EVO
 > 500GB.  My current OS sits on a 160GB drive so should be plenty.  I plan
 > to even add a boot image for the Gentoo LiveGUI thingy, maybe Knoppix or
 > something plus my usual OS.  By the way, caught one for sale for
 > $40.00.  It has a production date of 5/2021.
 >
 > My question is this.  Do I need anything special in the kernel or
 > special fstab options for this thing?  I know at one point there was
 > folks having problems with certain settings.  I did some googling and it
 > seems to be worked out but I want to be sure I don't blow this thing up
 > or something.
 >
 > Anything else that makes these special?  Any tips or tricks?
 >
 > Dale
 >
 > :-)  :-)
 >
 > P. S.  I'm hoping this will make my system a little more responsive.
 > Maybe.  Either way, that 160GB drive is getting a little full.
 >

Dale,
    I have 500GB SSDs and 1TB M.2 drives in all of my machines. No
machine boots from a spinning drive anyhmore. Never had any
problems.

    The only thing I've done differently is the errors=remount=ro item
below. Other than that if whatever OS you install sets up boot, and
the machine boots, then it's just a drive in my experience

Best wishes,
Mark

# / was on /dev/nvme1n1p3 during installation
UUID=3fe6798f-653f-42e8-8e96-7ba0d490bfdf /               ext4    
errors=remount-ro 0       1
# /boot/efi was on /dev/nvme0n1p1 during installation
UUID=60DF-9F56  /boot/efi       vfat    umask=0077      0       1


My 5-year old small box running 500GB SSD INTEL SSDSC2BF48 Atom processor, it 
is ON 24/7 running Asterisk and hylafax.
Never had a problem with it.
But it is recommended to to run via cron fstrim:
30 18 * * 2  /sbin/fstrim -v /

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Mark Knecht]

On Sat, Apr 15, 2023 at 3:47 PM Dale  wrote:
>
> Howdy,
>
> I finally broke down and bought a SSD.  It's a Samsung V-Nand 870 EVO
> 500GB.  My current OS sits on a 160GB drive so should be plenty.  I plan
> to even add a boot image for the Gentoo LiveGUI thingy, maybe Knoppix or
> something plus my usual OS.  By the way, caught one for sale for
> $40.00.  It has a production date of 5/2021.
>
> My question is this.  Do I need anything special in the kernel or
> special fstab options for this thing?  I know at one point there was
> folks having problems with certain settings.  I did some googling and it
> seems to be worked out but I want to be sure I don't blow this thing up
> or something.
>
> Anything else that makes these special?  Any tips or tricks?
>
> Dale
>
> :-)  :-)
>
> P. S.  I'm hoping this will make my system a little more responsive.
> Maybe.  Either way, that 160GB drive is getting a little full.
>

Dale,
   I have 500GB SSDs and 1TB M.2 drives in all of my machines. No
machine boots from a spinning drive anyhmore. Never had any
problems.

   The only thing I've done differently is the errors=remount=ro item
below. Other than that if whatever OS you install sets up boot, and
the machine boots, then it's just a drive in my experience

Best wishes,
Mark

# / was on /dev/nvme1n1p3 during installation
UUID=3fe6798f-653f-42e8-8e96-7ba0d490bfdf /   ext4
 errors=remount-ro 0   1
# /boot/efi was on /dev/nvme0n1p1 during installation
UUID=60DF-9F56  /boot/efi   vfatumask=0077  0   1

[gentoo-user] Finally got a SSD drive to put my OS on

[Dale]

Howdy,

I finally broke down and bought a SSD.  It's a Samsung V-Nand 870 EVO
500GB.  My current OS sits on a 160GB drive so should be plenty.  I plan
to even add a boot image for the Gentoo LiveGUI thingy, maybe Knoppix or
something plus my usual OS.  By the way, caught one for sale for
$40.00.  It has a production date of 5/2021. 

My question is this.  Do I need anything special in the kernel or
special fstab options for this thing?  I know at one point there was
folks having problems with certain settings.  I did some googling and it
seems to be worked out but I want to be sure I don't blow this thing up
or something. 

Anything else that makes these special?  Any tips or tricks? 

Dale

:-)  :-) 

P. S.  I'm hoping this will make my system a little more responsive. 
Maybe.  Either way, that 160GB drive is getting a little full.