Re: [gentoo-user] Re: system.map file in /boot. How to manage?

[Hayley]

On Thu, 2021-07-01 at 15:14 +, Grant Edwards wrote:
> On 2021-07-01, Neil Bothwick  wrote:
> 
> > make install names the files in a way that dracut and grub-mkconfig
> > recognise. Just run make install after make modules install. You've
> > let
> > the makefile copy all the other files, you may as well let it handle
> > the
> > final two :)
> 
> IIRC, "make install" requiers /sbin/installkernel -- an executable
> that's provided by one of installkernel-gentoo-3, debianutils, or
> installkernel-systemd-boot. Back in the day, a base installation
> didn't have /sbin/installkernel, and you needed to install it
> manually. Has that changed?
> 
> --
> Grant
> 
> 


The handbook says that sys-kernel/installkernel-gentoo should be
installed by default. [1]

Just throw in my 2 cents, I use genkernel to manage compiling and
installing the kernel. With a little configuring you can get it to
compile, install and run grub-mkconfig for you. That way I can't forget
to run grub-mkconfig or something.

Hayley

[1]
https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel#Installing_correct_installkernel

Re: [gentoo-user] system.map file in /boot. How to manage?

[William Kenworthy]

There is a lot online - basically its optional and needed just in case
(!) and should be versioned like its matching kernel and initrd.

BillK

one of many refs:
https://stackoverflow.com/questions/28936630/what-is-the-need-of-having-both-system-map-file-and-proc-kallsyms


On 1/7/21 2:59 pm, Dale wrote:
> Howdy,
>
> The subject line pretty much describes this.  How does one manage the
> system.map file in /boot?  Is it needed?  Should it be updated with each
> kernel?  I tend to keep 2 to 3 kernels installed.  I tend to keep 2 that
> I know are stable and one testing.  After a while, I may remove the
> oldest one and only have two, just in case.  Should I version the
> system.map file the same as kernels?  Does just one with no version get
> the job done?  Update the file with each kernel upgrade or install one
> and done?
>
> While at it, what does it even do?  If it needs it, it doesn't matter
> but just curious. 
>
> Thanks for any tips on this.
>
> Dale
>
> :-)  :-) 
>

[gentoo-user] system.map file in /boot. How to manage?

[Dale]

Howdy,

The subject line pretty much describes this.  How does one manage the
system.map file in /boot?  Is it needed?  Should it be updated with each
kernel?  I tend to keep 2 to 3 kernels installed.  I tend to keep 2 that
I know are stable and one testing.  After a while, I may remove the
oldest one and only have two, just in case.  Should I version the
system.map file the same as kernels?  Does just one with no version get
the job done?  Update the file with each kernel upgrade or install one
and done?

While at it, what does it even do?  If it needs it, it doesn't matter
but just curious. 

Thanks for any tips on this.

Dale

:-)  :-) 

Re: [gentoo-user] [OT] Using an odd number of drives in ZFS RaidZ

[Robert David]

Hi Frank,

On Tuesday, June 29, 2021 3:56:49 PM CEST Frank Steinmetzger wrote:
> Hello fellows
> 
> This is not really a Gentoo question, but at least my NAS (which this mail
> is about) is running Gentoo. :)
> 
> There are some people amongst this esteemed group that know their stuff
> about storage and servers and things, so I thought I might try my luck here.
> I’ve already looked on the Webs, but my question is a wee bit specific and
> I wasn’t able to find the exact answer (yet). And I’m a bit hesitant to ask
> this newbie-ish question in a ZFS expert forum. ;-)
> 
> Prologue:
> Due to how records are distributed across blocks in a parity-based ZFS vdev,
> it is recommended to use 2^n data disks. Technically, it is perfectly fine
> to deviate from it, but for performance reasons (mostly space efficiency)
> it is not the recommended way. That’s because the (default) maximum record
> size of 128 k itself is a power of 2 and thus can be distributed evenly on
> all drives. At least that’s my understanding. Is that correct?
> 
> So here’s the question:
> If I had three data drives, (c|w)ould I get around that problem by setting a
> record size that is divisible by 3, like 96 k, or even 3 M?

I would not bother with this. 128k is a good default for general usage
and even if you got 3 data disks the actual loss is pointless to think
about (assuming you got 4k disks).


> 
> 
> 
> Here’s the background of my question:
> Said NAS is based on a Mini-ITX case which has only four drive slots (which
> is the most common configuration for a case of this formfactor). I started
> with two 6 TB drives, running in a mirror configuration. One year later
> space was running out and I filled the remaining slots. To maximise
> reliability, I went with RaidZ2.
> 
> I reached 80 % usage (which is the recommended maximum for ZFS) and am
> now evaluating my options for the coming years.
> 1) Reduce use of space by re-encoding. My payload is mainly movies, among
>which are 3 TB of DVDs which can be shrunk by at least ⅔ by re-encoding.
>→ this takes time and computing effort, but is a long-term goal anyway.

I always think about in such cases if I really need such data. In many
cases with clear consideration I find out I may remove half of the data
without any pain. It is like cleaning my home, there are many things
extra and there is missing a space for real valuable things, with disk
data it is the same.

> 2) Replace all drives with bigger ones. There are three counter arguments:
>• 1000 € for four 10 TB drives (the biggest size available w/o helium)
>• they are only available with 7200 rpm (more power, noise and heat)
>• I am left with four perfectly fine 6 TB drives
> 3) Go for 4+2 RaidZ2. This requires a bigger case (with new PSU due to
>different form factor) and a SATA expansion card b/c the Mobo only has
>six connectors (I need at least one more for the system drive), costing
>250 € plus drives.
> 4) Convert to RaidZ1. Gain space of one drive at the cost of resilience. I
>can live with the latter; the server only runs occasionally and not for
>very long at a time. *** This option brings me to my question above,
>because it is easy to achieve and costs no €€€.

In any of my data arrays I have long time migrated off the RAIDZ to the
MIRROR or RAID10. You will find finally that the RAIDZ is slow and not
very flexible. Only think you gain is the extra space in constrained
array spaces. For RAID10 it is much easier to raise the size, just
resilvering to new bigger disks, removing old and expanding. The
resilvering speed is magnitude faster. And anyway much easier to recover
in cases of failure. 

If you really need the additional space, consider adding second jbod
with another disks.

Robert.

Re: [gentoo-user] system.map file in /boot. How to manage?

[Dr Rainer Woitok]

Dale,

On Thursday, 2021-07-01 01:59:57 -0500, you wrote:

> ...
> 
> The subject line pretty much describes this.  How does one manage the
> system.map file in /boot?  Is it needed?  Should it be updated with each
> kernel?  I tend to keep 2 to 3 kernels installed.

Same here.   And whenever  I configure  a new kernel  my kernel managing
script makes sure both,  the kernel I'm currenty running on  and the one
just configured are in "@world".  That way "emerge --depclean" will nev-
er remove a kernel package.

When I finally decide  to explicitly remove a kernel  my kernel managing
script will basically execute this before calling "grub-mkconfig":

   # Argument 1 must contain slot number of kernel to be removed.

   emerge --deselect --quiet sys-kernel/gentoo-sources:$1
   rm -fr /lib/modules/$1-gentoo /boot/*-$1-gentoo{,.old} \
  /usr/src/linux-$1-gentoo

This is stuff I somewhere picked up  from the Gentoo Handbook when I did
my first Gentoo install.

> ...
>  Should I version the
> system.map file the same as kernels?

Not sure about that.   Mine ARE versioned.   That's probably what "grub-
mkconfig" is doing by default.

Sincerely,
  Rainer

Re: [gentoo-user] system.map file in /boot. How to manage?

[tastytea]

On 2021-07-01 13:16+0200 Dr Rainer Woitok 
wrote:

> Dale,
> 
> On Thursday, 2021-07-01 01:59:57 -0500, you wrote:
> 
> > ...
> >  Should I version the
> > system.map file the same as kernels?  
> 
> Not sure about that.   Mine ARE versioned.   That's probably what
> "grub- mkconfig" is doing by default.

I compile kernels with `genkernel`, and it produces versioned files:
  System.map-5.4.97-gentoo-x86_64
  System.map-5.10.27-gentoo-x86_64
  initramfs-5.4.97-gentoo-x86_64.img
  initramfs-5.10.27-gentoo-x86_64.img
  vmlinuz-5.4.97-gentoo-x86_64
  vmlinuz-5.10.27-gentoo-x86_64

Kind regards, tastytea

-- 
Get my PGP key with `gpg --locate-keys tasty...@tastytea.de` or at
.


pgpoUJvhtvwyz.pgp
Description: Digitale Signatur von OpenPGP

Re: [gentoo-user] system.map file in /boot. How to manage?

[Dale]

tastytea wrote:
> On 2021-07-01 13:16+0200 Dr Rainer Woitok 
> wrote:
>
>> Dale,
>>
>> On Thursday, 2021-07-01 01:59:57 -0500, you wrote:
>>
>>> ...
>>>  Should I version the
>>> system.map file the same as kernels?  
>> Not sure about that.   Mine ARE versioned.   That's probably what
>> "grub- mkconfig" is doing by default.
> I compile kernels with `genkernel`, and it produces versioned files:
>   System.map-5.4.97-gentoo-x86_64
>   System.map-5.10.27-gentoo-x86_64
>   initramfs-5.4.97-gentoo-x86_64.img
>   initramfs-5.10.27-gentoo-x86_64.img
>   vmlinuz-5.4.97-gentoo-x86_64
>   vmlinuz-5.10.27-gentoo-x86_64
>
> Kind regards, tastytea
>


This info is helpful.  I tend to do mine by hand, I may only upgrade my
kernel once, maybe twice, a year.  I don't reboot very often so there is
no point in upgrading every time a new one comes out.  After reading
this, I've copied over the proper system.map files and versioned them to
match the kernel names.  I'll run grub-mkconfig just in case shortly. 
I've always had *A* file there but never copied newer versions over
before. Honestly, I didn't know if it was needed or not.  It has a lot
of info in it tho so I figured it wise to ask. 

Thanks to all for the info.  Now I have a better understanding of what I
need to do and how.

Dale

:-)  :-) 

P. S.  Back to the woods.  Putting out fertilizer for the trees. 
Supposed to rain in the next day or so.

Re: [gentoo-user] system.map file in /boot. How to manage?

[Neil Bothwick]

On Thu, 1 Jul 2021 13:16:29 +0200, Dr Rainer Woitok wrote:

> Same here.   And whenever  I configure  a new kernel  my kernel managing
> script makes sure both,  the kernel I'm currenty running on  and the one
> just configured are in "@world".  That way "emerge --depclean" will nev-
> er remove a kernel package.

You can also do that with sets. Add this to sets.conf

[kernels]
class = portage.sets.dbapi.OwnerSet
world-candidate = False
files = /usr/src

add emerge -n @kernels once. Then depclean will never touch a kernel
source package.

> > ...
> >  Should I version the
> > system.map file the same as kernels?  
> 
> Not sure about that.   Mine ARE versioned.   That's probably what "grub-
> mkconfig" is doing by default.

grub-mkconfig only reads the files, it is the make install step of kernel
installation that takes care of copying the files to /boot with the
correct version numbers. ISTR Dale prefers to copy the kernel files
manually, which is why his System.map is not versioned.


-- 
Neil Bothwick

Suborbital Ballistic-Propulsion Engineer
Not Exactly A Rocket Scientist


pgpQ5dtdU3GZs.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] system.map file in /boot. How to manage?

[Dale]

Neil Bothwick wrote:
> On Thu, 1 Jul 2021 13:16:29 +0200, Dr Rainer Woitok wrote:
>
>> Same here.   And whenever  I configure  a new kernel  my kernel managing
>> script makes sure both,  the kernel I'm currenty running on  and the one
>> just configured are in "@world".  That way "emerge --depclean" will nev-
>> er remove a kernel package.
> You can also do that with sets. Add this to sets.conf
>
> [kernels]
> class = portage.sets.dbapi.OwnerSet
> world-candidate = False
> files = /usr/src
>
> add emerge -n @kernels once. Then depclean will never touch a kernel
> source package.
>
>>> ...
>>>  Should I version the
>>> system.map file the same as kernels?  
>> Not sure about that.   Mine ARE versioned.   That's probably what "grub-
>> mkconfig" is doing by default.
> grub-mkconfig only reads the files, it is the make install step of kernel
> installation that takes care of copying the files to /boot with the
> correct version numbers. ISTR Dale prefers to copy the kernel files
> manually, which is why his System.map is not versioned.
>
>


I do copy mine manually.  It's how it was done when I first started
using Gentoo and I just stuck with it, it works.  It's just one
additional file.  Thing is, I may read up on the script way of doing
things.  It may help make dracut and grub-mkconfig happy.  Sometimes I
have to fiddle with names to get them both happy.  I suspect the script
tools handle that with ease. 

Now I got to go find that signature about the cat and free bagpipes.  I
want to send that to a friend.  She'll get a kick out of that.  lol 

Dale

:-)  :-) 

Re: [gentoo-user] [OT] Using an odd number of drives in ZFS RaidZ

[antlists]

On 01/07/2021 14:47, Robert David wrote:

Hi Frank,





In any of my data arrays I have long time migrated off the RAIDZ to the
MIRROR or RAID10. You will find finally that the RAIDZ is slow and not
very flexible. Only think you gain is the extra space in constrained
array spaces. For RAID10 it is much easier to raise the size, just
resilvering to new bigger disks, removing old and expanding. The
resilvering speed is magnitude faster.



And anyway much easier to recover
in cases of failure.


ARE YOU SURE???

The standard mirror does not cope with corruption very well. Lose a disk 
and resilvering is fast. Corrupt the data, and you'll be tearing your 
hair out why things go wrong randomly, with no automated way, even once 
you've realised what's happened, to recover your data other than a 
restore from backup.



If you really need the additional space, consider adding second jbod
with another disks.


That'd be my approach - migrate a load of stuff off onto another disk 
elsewhere, but that's not what the OP wants to do.


Robert.


Cheers,
Wol

Re: [gentoo-user] system.map file in /boot. How to manage?

[Neil Bothwick]

On Thu, 1 Jul 2021 09:41:27 -0500, Dale wrote:

> > grub-mkconfig only reads the files, it is the make install step of
> > kernel installation that takes care of copying the files to /boot
> > with the correct version numbers. ISTR Dale prefers to copy the
> > kernel files manually, which is why his System.map is not versioned.

> I do copy mine manually.  It's how it was done when I first started
> using Gentoo and I just stuck with it, it works.  It's just one
> additional file.  Thing is, I may read up on the script way of doing
> things.  It may help make dracut and grub-mkconfig happy.  Sometimes I
> have to fiddle with names to get them both happy.  I suspect the script
> tools handle that with ease. 

make install names the files in a way that dracut and grub-mkconfig
recognise. Just run make install after make modules install. You've let
the makefile copy all the other files, you may as well let it handle the
final two :)


-- 
Neil Bothwick

By the time you can make ends meet, they move the ends.


pgpvs_sHb1ci2.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] [OT] Using an odd number of drives in ZFS RaidZ

[antlists]

On 01/07/2021 00:31, Frank Steinmetzger wrote:

Antlist made a similar suggestion using external USB, and I gave a more
detailed answer in reply to his mail.


I've got this ...

https://www.amazon.co.uk/gp/product/B072J52TR1/ref=ppx_yo_dt_b_asin_title_o04_s00?ie=UTF8=1

It's eSATA not USB.

It's worked fine for me, but no I haven;t used it much - I shall 
probably be using it a bit very soon as I finish building my new system...


Cheers,
Wol

Re: [gentoo-user] system.map file in /boot. How to manage?

[Dr Rainer Woitok]

Neil,

On Thursday, 2021-07-01 15:28:51 +0100, you wrote:

> On Thu, 1 Jul 2021 13:16:29 +0200, Dr Rainer Woitok wrote:
> 
> > Same here.   And whenever  I configure  a new kernel  my kernel managing
> > script makes sure both,  the kernel I'm currenty running on  and the one
> > just configured are in "@world".  That way "emerge --depclean" will nev-
> > er remove a kernel package.
> 
> You can also do that with sets. Add this to sets.conf
> 
> [kernels]
> class = portage.sets.dbapi.OwnerSet
> world-candidate = False
> files = /usr/src
> 
> add emerge -n @kernels once. Then depclean will never touch a kernel
> source package.

Nice trick, thank you :-).   This would slightly simplify my kernel man-
aging script.

Sincerely,
  Rainer

Re: [gentoo-user] system.map file in /boot. How to manage?

[antlists]

On 01/07/2021 15:41, Dale wrote:

I do copy mine manually.  It's how it was done when I first started
using Gentoo and I just stuck with it, it works.  It's just one
additional file.


I copied my kernels manually to start with. Then I discovered "make 
install". (and "make modules_install"). Much simpler.


And you don't need to muck about with grub.cfg by hand either. I think 
I'm going to switch to grub-mkconfig, because you can always output it 
to a new file to sanity-check before you send it live (and back up the 
old one, so you can always recover ...).


Cheers,
Wol

[gentoo-user] Re: system.map file in /boot. How to manage?

[Grant Edwards]

On 2021-07-01, Neil Bothwick  wrote:

> make install names the files in a way that dracut and grub-mkconfig
> recognise. Just run make install after make modules install. You've let
> the makefile copy all the other files, you may as well let it handle the
> final two :)

IIRC, "make install" requiers /sbin/installkernel -- an executable
that's provided by one of installkernel-gentoo-3, debianutils, or
installkernel-systemd-boot. Back in the day, a base installation
didn't have /sbin/installkernel, and you needed to install it
manually. Has that changed?

--
Grant

Re: [gentoo-user] [OT] Using an odd number of drives in ZFS RaidZ

[Frank Steinmetzger]

Am Thu, Jul 01, 2021 at 04:07:39PM +0100 schrieb antlists:
> On 01/07/2021 00:31, Frank Steinmetzger wrote:
> > Antlist made a similar suggestion using external USB, and I gave a more
> > detailed answer in reply to his mail.
> 
> I've got this ...
> 
> https://www.amazon.co.uk/gp/product/B072J52TR1/ref=ppx_yo_dt_b_asin_title_o04_s00?ie=UTF8=1
> 
> It's eSATA not USB.

Which is very similar in function to the one I have, only mine is for
installation in a 5¼″ case slot. When I bought it years ago I was thing
about a desktop dock like yours, because that can also be used with my
laptop. But I found it neater to install it cleanly into my PC case. :-)


> It's worked fine for me, but no I haven;t used it much - I shall probably be
> using it a bit very soon as I finish building my new system...

When I was reading up on the differences between SAS and SATA (because some
enclosures and cases use a SAS backplane), one of the items was multipath
support: SAS can address several drives, SATA only one. So eSATA is limited
to a single drive per cable.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Arrogance is the art of being proud of one’s own stupidity.


signature.asc
Description: PGP signature

Re: [gentoo-user] [OT] Using an odd number of drives in ZFS RaidZ

[Frank Steinmetzger]

Am Thu, Jul 01, 2021 at 04:01:25PM +0100 schrieb antlists:

> On 01/07/2021 14:47, Robert David wrote:
> > Hi Frank,
> > 
> 
> > 
> > In any of my data arrays I have long time migrated off the RAIDZ to the
> > MIRROR or RAID10. You will find finally that the RAIDZ is slow and not
> > very flexible.

Flexibility indeed. This bites me in the butt now. But performance is
sufficient for me, because everything can saturate gigabit ethernet and
there are no VMs involved.

A scrub currently takes 10½ hours. Considering each drive is filled with
6 TB * 80 % = 4.8 TB, that’s an average of 130 MB/s/device which seems not
so bad for 5400 rpm drives.

When I installed drives #3 and 4, I thought long and hard about whether to
use Raid-10 or Z2. The increased resilience won the argument (any 2 drives
over 2 particular drives).

> > If you really need the additional space, consider adding second jbod
> > with another disks.
> 
> That'd be my approach - migrate a load of stuff off onto another disk
> elsewhere, but that's not what the OP wants to do.

Yeah… I know that for some people, carrying around TBs of movies and TV
series is overkill, but I like having them, and I like having them all in
this neat little box:
https://www.inter-tech.de/products/ipc/storage-cases/sc-4100
  :)

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

“If I could explain it to the average person, I wouldn't have been worth
the Nobel Prize.” – Richard Feynman


signature.asc
Description: PGP signature