Re: Modular raidframe
On Sun 20 Dec 2015 at 16:19:34 +0800, Paul Goyette wrote: > I like to think that I run a kernel from a third class: > > HIGHLY-MODULAR > > These kernels include only a minimum amount of built-in code, > and any additional functionality is loaded as needed. I have > fewer than 20 built-in modules, and generally run with only > 40 modules total (the difference being loaded dynamically). How does one most easily create a kernel of this type (and know which parts are loadable and which are not)? I've been making my own kernels by starting with GENERIC and removing things from it that I don't have or need. Is there a simple way to see which of those things are also modules (so they can come back if I need them after all)? The names in /stand/modules don't always seem to match 100% with words from the config file. -Olaf. -- ___ Olaf 'Rhialto' Seibert -- The Doctor: No, 'eureka' is Greek for \X/ rhialto/at/xs4all.nl-- 'this bath is too hot.' signature.asc Description: PGP signature
Re: Modular raidframe
On Sun, 20 Dec 2015, Rhialto wrote: On Sun 20 Dec 2015 at 16:19:34 +0800, Paul Goyette wrote: I like to think that I run a kernel from a third class: HIGHLY-MODULAR These kernels include only a minimum amount of built-in code, and any additional functionality is loaded as needed. I have fewer than 20 built-in modules, and generally run with only 40 modules total (the difference being loaded dynamically). How does one most easily create a kernel of this type (and know which parts are loadable and which are not)? I've been making my own kernels by starting with GENERIC and removing things from it that I don't have or need. Is there a simple way to see which of those things are also modules (so they can come back if I need them after all)? The names in /stand/modules don't always seem to match 100% with words from the config file. The stuff in /stand//modules is pretty close to what is in the config file, close enough that you should be able to guess with > 90% accuracy. :) Device driver modules for actual devices (those with a bmajor or cmajor) must match names for autoload to work. File systems also match. Emulation/exec modules are almost always exec_* or compat_* The ones that don't always match are those that provide optional system calls. You can examine src/sys/kern/syscalls.master to see which syscalls are option, which option they depend on, and which module they are contained in. This should get you very close to 100%. +--+--++ | Paul Goyette | PGP Key fingerprint: | E-mail addresses: | | (Retired)| FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com | | Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org | +--+--++
Re: Modular raidframe
On Sun, 20 Dec 2015, Michael van Elst wrote: p...@whooppee.com (Paul Goyette) writes: The attached diffs (plus 2 new files) convert our existing raidframe driver into a loadable module. Does this still work with a non-MODULAR kernel? Short answer: Yes More details: Whether or not a kernel is built with "options MODULAR", it still contains modules. The only thing that the option controls is whether or not the in-kernel linker is included. Code that has been modularized is included as a built-in module (and initialized as such) regardless of the option. So the classes of kernels are: non-MODULAR - "no options MODULAR" These kernels have a static configuration, defined at build time. They cannot be extended at boot- or run-time since there is no linker included. Built-in modules are still treated as modules, initialized by module_init() (called from init_main()), etc. And built-in modules can still be disabled with the modunload(8) command. MODULAR - "option MODULAR" These kernels include an in-kernel linker which can be used to add additional code at run-time. Typical GENERIC kernels generally fall into this class, even though nearly everything is already linked in at build time. On -current amd64, we include almost 150 built-in modules. I like to think that I run a kernel from a third class: HIGHLY-MODULAR These kernels include only a minimum amount of built-in code, and any additional functionality is loaded as needed. I have fewer than 20 built-in modules, and generally run with only 40 modules total (the difference being loaded dynamically). +--+--++ | Paul Goyette | PGP Key fingerprint: | E-mail addresses: | | (Retired)| FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com | | Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org | +--+--++
Modular raidframe
I had a situation the other day where I wanted to recover some data that
was sitting on an old decommissioned raidset. Since my current kernel
doesn't have raid installed, I had to build a new kernel (GENERIC was
fine), shutdown the production box, boot up the new kernel, install the
disk drive, mount the raidset, restore files, shutdown, and reboot
normally.
What a pain in the neck. Wouldn't it have been so much simpler if I
could have just attached the drive, mounted the raidset (including
autoload of the driver module, retrieved my data, and perhaps unloaded
the no-longer-needed module? :) I had looked at doing this previously,
but backed away because it "felt like" too much complex work.
It turns out not to be such a difficult task after all!
The attached diffs (plus 2 new files) convert our existing raidframe
driver into a loadable module. Most of the changes made were needed for
enabling module unload (detaching units and releasing data structures
dynamically, rather than waiting for system shutdown time).
I've done some fairly extensive testing, on both "bare-iron" and qemu
virtual machines, and it seems to work just fine. Including scanning
for auto-configurable raidsets when the module is loaded. I've also
tested the built-in variant of the module to ensure that that still
works, too.
I'm planning to commit this later this week. I'd appreciate any review
and (hopefully constructive) comments that the community might offer.
+--+--++
| Paul Goyette | PGP Key fingerprint: | E-mail addresses: |
| (Retired)| FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com |
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org |
+--+--++# $NetBSD$
ioconf raidframe
include "conf/files"
pseudo-device raid
# $NetBSD$
.include "../Makefile.inc"
IOCONF= raid.ioconf
.PATH: ${S}/dev/raidframe
KMOD= raid
SRCS+= rf_acctrace.c rf_alloclist.c rf_aselect.c
SRCS+= rf_callback.c rf_chaindecluster.crf_copyback.c
SRCS+= rf_cvscan.c rf_dagdegrd.c rf_dagdegwr.c
SRCS+= rf_dagffrd.crf_dagffwr.c rf_dagfuncs.c
SRCS+= rf_dagutils.c rf_debugMem.c rf_debugprint.c
SRCS+= rf_decluster.c rf_declusterPQ.c rf_diskqueue.c
SRCS+= rf_disks.c rf_driver.crf_engine.c
SRCS+= rf_evenodd.crf_evenodd_dagfuncs.c rf_evenodd_dags.c
SRCS+= rf_fifo.c rf_interdecluster.crf_invertq.c
SRCS+= rf_layout.c rf_map.c rf_mcpair.c
SRCS+= rf_netbsdkintf.crf_nwayxor.c rf_options.c
SRCS+= rf_paritylog.c rf_paritylogDiskMgr.c rf_paritylogging.c
SRCS+= rf_parityloggingdags.c rf_paritymap.c rf_parityscan.c
SRCS+= rf_pq.c rf_pqdeg.c rf_pqdegdags.c
SRCS+= rf_psstatus.c rf_raid0.c rf_raid1.c
SRCS+= rf_raid4.c rf_raid5.c rf_raid5_rotatedspare.c
SRCS+= rf_reconbuffer.crf_reconmap.c rf_reconstruct.c
SRCS+= rf_reconutil.c rf_revent.crf_shutdown.c
SRCS+= rf_sstf.c rf_states.crf_stripelocks.c
SRCS+= rf_strutils.c rf_utils.c rf_compat50.c
CPPFLAGS+= -DRAID_AUTOCONFIG=1
# Include optional raid styles
CPPFLAGS+= -DRF_INCLUDE_EVENODD=1
CPPFLAGS+= -DRF_INCLUDE_RAID5_RS=1
CPPFLAGS+= -DRF_INCLUDE_PARITYLOGGING=1
CPPFLAGS+= -DRF_INCLUDE_CHAINDECLUSTER=1
CPPFLAGS+= -DRF_INCLUDE_INTERDECLUSTER=1
CPPFLAGS+= -DRF_INCLUDE_PARITY_DECLUSTERING=1
CPPFLAGS+= -DRF_INCLUDE_PARITY_DECLUSTERING_DS=1
CPPFLAGS+= -DCOMPAT_50
.include
Index: distrib/sets/lists/modules/mi
===
RCS file: /cvsroot/src/distrib/sets/lists/modules/mi,v
retrieving revision 1.82
diff -u -p -r1.82 mi
--- distrib/sets/lists/modules/mi 7 Dec 2015 11:38:46 - 1.82
+++ distrib/sets/lists/modules/mi 20 Dec 2015 04:26:50 -
@@ -206,6 +206,8 @@
./@MODULEDIR@/puffs/puffs.kmod base-kernel-modules kmod
./@MODULEDIR@/putter base-kernel-modules kmod
./@MODULEDIR@/putter/putter.kmod base-kernel-modules kmod
+./@MODULEDIR@/raid base-kernel-modules kmod
+./@MODULEDIR@/raid/raid.kmod base-kernel-modules kmod
./@MODULEDIR@/scsiverbose base-kernel-modules kmod
./@MODULEDIR@/scsiverbose/scsiverbose.kmod base-kernel-modules kmod
./@MODULEDIR@/sdt base-obsolete obsolete
Index: sys/modules/Makefile
===
RCS
Re: Modular raidframe
p...@whooppee.com (Paul Goyette) writes: >The attached diffs (plus 2 new files) convert our existing raidframe >driver into a loadable module. Does this still work with a non-MODULAR kernel? -- -- Michael van Elst Internet: mlel...@serpens.de "A potential Snark may lurk in every tree."
