Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
Jonas Meurer jo...@freesources.org wrote:
i guess the best solution would be to improve the module dependency
system. cipher meta-modules like 'aes' should depend on all available
implementations. same for ivciphers, hashs, rng implementations etc.
We already do that for algorithms at least, they should all have
aliases of the form ALG or ALG-all.
Things like chainiv/eseqiv should just be included always. There
are unlikely to be any new algorithms of that type.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmVHI~} herb...@gondor.apana.org.au
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line unsubscribe linux-crypto in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
* Jonas Meurer | 2009-09-01 02:51:20 [+0200]:
as you already discovered, that's because the initramfs hook script for
cryptroot adds some crypto modules manually in debian. currently these
are dm-mod, dm-crypt, cbc, aes, sha256 and all the modules they depend
on.
You could add something like
|for dev in $rootdev $resumedevs; do
| info=$(cryptsetup luksDump $rootdev)
| if [ $? -ne 0]; then
| # Partition is now encrypted
| continue
| fi
| cipher=$(cryptsetup luksDump /dev/md1 |grep Cipher name: | \
|sed 's...@^cipher name:\s\+@@')
| add_crypto_modules $cipher
| mode=$(cryptsetup luksDump /dev/md1 |grep Cipher name:)
| echo $mode | sed \
| -e 's...@^cipher name:\s\+@@') \
| -e 's...@-@ @' \
| -e 's@:@ @'
| # mode is now block_mode IV_mode hash
| blk_mode=$(echo $mode | awk '{print $1}'
| hash=$(echo $mode | awk '{print $3})'
| add_crypto_modules $blk_mode
| add_crypto_modules $hash
|done
instead of hardcoding cbc, aes sha256
So this brings me to the following question: Why are you so picky and you
don't take all of the modules? sh 2.6.30-1 on various debian archs:
du -sh x86_64 i386 powerpc s390x mips(r5k-ip32)
crypto 720K636K 1.1M 804K968K
arch/arch/crypto 72K 36K 092K 0
driver/crypto 32K 60K 60K 0 52K
Total 824K732K 1.2M 896K1020
So all modules together have an average footprint of 941.0K. This isn't
that much. There is no size limit on initrd unless you are on a system
with has 32MiB or something like that :)
With *all* those crypto modules which are selected and built you are
able to cover all corner cases.
I took a look at hooks/cryptroot and really and only a few modules are
included. Right now you don't include HW cipher e.g. VIA's padlock or
AMD's geode. If you don't load those module while openning the root
partition (the first request), then the module is never loaded / used.
The other unhandled case are non default ciphers like xts(aes) which one
might use. The brand new aes implementation aesni-intel is also not be
considered. Most modules have _generic in their name if there is
another implementation available and name-arch if there is an assembly
variant but I would prefer that this does not become part of an ABI
where one can rely on.
Would it be okay for you to change the way the crypto modules are picked
and include the full set?
i don't like the idea to add _all_ crypto modules into the initramfs per
default. the size of crypto and arch/arch/crypto will keep on growing
for kernels which do have ship all available modules for ciphers and
implementations. i think that the initramfs should be a _minimal_
system, and detecting the required crypto modules should be possible.
Why is a minimal initramsfs here a good thing here? 1 MiB isn't that
huge and with gzip it does not grow much. Here is an example on my amd64
with current initramfs as of 2.6.30-1-amd64:
picky : 10511917 bytes / 11 MiB
full crypto : 10729154 bytes / 11 MiB / + 2.1%
So. On AMD64 we are talking here about 200KiB or 2% of the whole image
size.
If arch/*/crypto grows than just because there is more support and this
is a good thing not bad.
alternative crypto cipher and blockcipher modules are already detected
if required, as information about these are available from the cryptroot
configuration (either luks header or cmdline options given to cryptsetup).
ah so my little script was not required. What about d-i? AFAIK that one
runs from initramfs and relies on modules which are selected here. Isn't
this the case?
i added cryptomgr, chainiv and krng to the list of required modules in
/usr/share/initramfs-tools/hooks/cryptroot for now.
And in .32 you have work again because you need eseqiv.
still i see the point that this solution has it's drawbacks. modules for
crypto hardware and cipher implementations not following the convention to
call modules cipher_generic and cipher-alias aren't detected by the
current system.
exactly. Plus runtime dependency.
i guess the best solution would be to improve the module dependency
system. cipher meta-modules like 'aes' should depend on all available
implementations. same for ivciphers, hashs, rng implementations etc.
what do you think about it?
I don't the like extra work for 2%. Even if someone does this, you still
have lets say the geode, via-padlock and the hfin driver on a 486 system
which can't use all of them and relies on pure SW. No benefit there.
Okay. You put the crypto modules in initramfs in case the root partition
and or swap is encrypted. In case just /data is encrypted you don't need
those modules at all. So d-i needs all them in case the user might want
to xts(twofish) and not just the standard thing. The user-initramfs
script could take the full set of crypto
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
hey, On 30/08/2009 Sebastian Andrzej Siewior wrote: * Sebastian Andrzej Siewior | 2009-08-28 10:00:56 [+0200]: the problem is not reproducible with a debian/unstable 2.6.30.6 kernel, even though it has cbc compiled as module as well. but if I recompile the same kernel sources with Celejars kernel .config, the problem occurs. thus it must be related to the kernel config in some way. It must be the kernel confing since I run .30.stable and it works. I try to look at it later. Your kernel config is fine, the problem is that the initramfs tools do not copy all of the required modules into the initramfs. The missing modles are: - cryptomgr: that one is responsible to load the cbc and aes module and bind them to cbc(aes) - chainiv: that one creates IVs if the user does not specify one. dm-crypt probably does not use that one but is required due to the way crypto works atm. - krng: provides random numbers and is required by chainiv. If you add those three to /etc/initramfs/modules than it should work. Could someone please look at initramfs to figure out why those three modules are not copied in this reduced setup? the reason is simply that no other crypto modules define depends on the listed ones: # modinfo -F depends dm-crypt dm-mod,crypto_blkcipher # modinfo -F aes_generic sha256_generic cbc crypto_algapi crypto_hash crypto_algapi,crypto_blkcipher # modinfo -F crypto_blkcipher crypto_hash crypto_algapi crypto_algapi and even the new modules don't depend on each other: # modinfo -F cryptomgr crypto_hash,crypto_algapi,crypto_blkcipher,aead,pcompress # modinfo -F chainiv crypto_algapi,rng,crypto_wq,crypto_blkcipher # modinfo -F krng rng,crypto_algapi so the following depends should be added/changed: - chainiv should depend o 'krng' instead of 'rng' at least - maybe cipher modules like aes,serpent,... should depend on 'cryptomgr' instead of 'crypto_algapi' - crypto_algapi should depend on chainiv these changes are pure guesses, i don't know the details. but at least additional depends need to be defined for crypto modules, don't you think so? greetings, jonas -- To unsubscribe from this list: send the line unsubscribe linux-crypto in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
reopen 541835 thanks hey, On 31/08/2009 Sebastian Andrzej Siewior wrote: * Jonas Meurer | 2009-08-31 17:52:00 [+0200]: Could someone please look at initramfs to figure out why those three modules are not copied in this reduced setup? the reason is simply that no other crypto modules define depends on the listed ones: Well yes, but aes cbc is placed in the new initrd. I took a look, read below. as you already discovered, that's because the initramfs hook script for cryptroot adds some crypto modules manually in debian. currently these are dm-mod, dm-crypt, cbc, aes, sha256 and all the modules they depend on. so the following depends should be added/changed: - chainiv should depend o 'krng' instead of 'rng' at least rng is a subsubsystem within crypto. krng is one implementation which provides random numbers. So does ansi_cprng. Another implementation may come. - maybe cipher modules like aes,serpent,... should depend on 'cryptomgr' instead of 'crypto_algapi' This isn't possible without dummy variables because they don't need each other in first place. We could move code but crypto_algapi can live without cryptomgr. However cryptomgr doesn't make sense without crypto_algapi. Lets say you have a VIA CPU which provides cbc(aes) in HW. So you don't need cryptomgr because cbc(aes) is allready available. You need aes_generic but that's another story :) - crypto_algapi should depend on chainiv chainiv is one possible implementation eseqiv is another one. The later is used on async-hw and in 2.6.32+ on SMP afaik. these changes are pure guesses, i don't know the details. but at least additional depends need to be defined for crypto modules, don't you think so? We have them in Kconfig. All required modules are built and are available. However the things aren't that easy. The crypto subsystem is very flexible and undocumented and this is the problem here I think. ok, i see the point that it's not enough to simply add some additional dependencies in order to fix the problem. seems like an easy solution doesn't exist. So this brings me to the following question: Why are you so picky and you don't take all of the modules? sh 2.6.30-1 on various debian archs: du -sh x86_64 i386 powerpc s390x mips(r5k-ip32) crypto 720K636K 1.1M 804K968K arch/arch/crypto 72K 36K 092K 0 driver/crypto 32K 60K 60K 0 52K Total 824K732K 1.2M 896K1020 So all modules together have an average footprint of 941.0K. This isn't that much. There is no size limit on initrd unless you are on a system with has 32MiB or something like that :) With *all* those crypto modules which are selected and built you are able to cover all corner cases. I took a look at hooks/cryptroot and really and only a few modules are included. Right now you don't include HW cipher e.g. VIA's padlock or AMD's geode. If you don't load those module while openning the root partition (the first request), then the module is never loaded / used. The other unhandled case are non default ciphers like xts(aes) which one might use. The brand new aes implementation aesni-intel is also not be considered. Most modules have _generic in their name if there is another implementation available and name-arch if there is an assembly variant but I would prefer that this does not become part of an ABI where one can rely on. Would it be okay for you to change the way the crypto modules are picked and include the full set? i don't like the idea to add _all_ crypto modules into the initramfs per default. the size of crypto and arch/arch/crypto will keep on growing for kernels which do have ship all available modules for ciphers and implementations. i think that the initramfs should be a _minimal_ system, and detecting the required crypto modules should be possible. alternative crypto cipher and blockcipher modules are already detected if required, as information about these are available from the cryptroot configuration (either luks header or cmdline options given to cryptsetup). i added cryptomgr, chainiv and krng to the list of required modules in /usr/share/initramfs-tools/hooks/cryptroot for now. still i see the point that this solution has it's drawbacks. modules for crypto hardware and cipher implementations not following the convention to call modules cipher_generic and cipher-alias aren't detected by the current system. i guess the best solution would be to improve the module dependency system. cipher meta-modules like 'aes' should depend on all available implementations. same for ivciphers, hashs, rng implementations etc. what do you think about it? greetings, jonas signature.asc Description: Digital signature
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
* Sebastian Andrzej Siewior | 2009-08-28 10:00:56 [+0200]: the problem is not reproducible with a debian/unstable 2.6.30.6 kernel, even though it has cbc compiled as module as well. but if I recompile the same kernel sources with Celejars kernel .config, the problem occurs. thus it must be related to the kernel config in some way. It must be the kernel confing since I run .30.stable and it works. I try to look at it later. Your kernel config is fine, the problem is that the initramfs tools do not copy all of the required modules into the initramfs. The missing modles are: - cryptomgr: that one is responsible to load the cbc and aes module and bind them to cbc(aes) - chainiv: that one creates IVs if the user does not specify one. dm-crypt probably does not use that one but is required due to the way crypto works atm. - krng: provides random numbers and is required by chainiv. If you add those three to /etc/initramfs/modules than it should work. Could someone please look at initramfs to figure out why those three modules are not copied in this reduced setup? Sebastian -- To unsubscribe from this list: send the line unsubscribe linux-crypto in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
On Sun, 30 Aug 2009 17:37:22 +0200 Sebastian Andrzej Siewior sebast...@ml.breakpoint.cc wrote: * Sebastian Andrzej Siewior | 2009-08-28 10:00:56 [+0200]: the problem is not reproducible with a debian/unstable 2.6.30.6 kernel, even though it has cbc compiled as module as well. but if I recompile the same kernel sources with Celejars kernel .config, the problem occurs. thus it must be related to the kernel config in some way. It must be the kernel confing since I run .30.stable and it works. I try to look at it later. Your kernel config is fine, the problem is that the initramfs tools do not copy all of the required modules into the initramfs. The missing modles are: - cryptomgr: that one is responsible to load the cbc and aes module and bind them to cbc(aes) - chainiv: that one creates IVs if the user does not specify one. dm-crypt probably does not use that one but is required due to the way crypto works atm. - krng: provides random numbers and is required by chainiv. If you add those three to /etc/initramfs/modules than it should work. Right your are! It now works, thanks very much. I guess the catch was that the working (stock Debian) kernel config built cryptomgr (CRYPTO_MANAGER and CRYPTO_MANAGER2) and krng (CRYPTO_RNG2, although not CRYPTO_RNG) and chainiv (CRYPTO_BLKCIPHER2, although not CRYPTO_BLKCIPHER - I have the impression that chainiv is now controlled by the BLKCIPHER options?) into the kernel, not as modules, so I wasn't hit by the initramfs failure until I built them as modules (I have a bit of a fetish for building everything as a module, especially when using initrd's :/). Could someone please look at initramfs to figure out why those three modules are not copied in this reduced setup? Celejar -- mailmin.sourceforge.net - remote access via secure (OpenPGP) email ssuds.sourceforge.net - A Simple Sudoku Solver and Generator -- To unsubscribe from this list: send the line unsubscribe linux-crypto in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
* Randy Dunlap | 2009-08-27 12:34:01 [-0700]: On 27/08/2009 Randy Dunlap wrote: On Tue, 25 Aug 2009 19:58:52 -0400 Celejar wrote: I'm having a pretty bizarre problem with my kernel crypto configuration. I need support for a bog standard LUKS (aes / cbc-essiv:sha256) / cryptsetup installation, but even after I enable virtually everything in the crypto section of the kernel configs, cbc fails to load. All the relevant modules are exist (dm-mod, dm-crypt, crypto_blkcipher, crypto_algapi, crypto_hash, aes_generic, sha256_generic), but even after modprobing / insmoding everything, /proc/crypto shows that aes and sha is there, but not cbc. The problem has been reproduced (using my kernel config) by Jonas Meurer, the Debian cryptsetup maintainer, so it's not just me ;). We've tried numerous different kernel versions in the .30 / .31 range. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=541835 Does this mean that something else somewhere in the kernel needs to be configured but isn't, and the necessary dependency isn't properly declared? My config is at: http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=45;filename=config-2.6.31-rc6-lizzie-00042-gb2add73;att=1;bug=541835 [I'm not subscribed to lkml; please cc me on responses] the problem rather is that loading the cbc blockcipher module simply does nothing. the module is listed in /proc/modules, but the blockcipher is still missing from /proc/crypto. Yes. cbc(aes) is auto-generated once someone requests such a cipher. The block modes are not listed. the problem is not reproducible with a debian/unstable 2.6.30.6 kernel, even though it has cbc compiled as module as well. but if I recompile the same kernel sources with Celejars kernel .config, the problem occurs. thus it must be related to the kernel config in some way. It must be the kernel confing since I run .30.stable and it works. I try to look at it later. --- ~Randy Sebastian -- To unsubscribe from this list: send the line unsubscribe linux-crypto in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
On Thu, 27 Aug 2009 20:35:01 +0200 Jonas Meurer wrote: hey, On 27/08/2009 Randy Dunlap wrote: On Tue, 25 Aug 2009 19:58:52 -0400 Celejar wrote: I'm having a pretty bizarre problem with my kernel crypto configuration. I need support for a bog standard LUKS (aes / cbc-essiv:sha256) / cryptsetup installation, but even after I enable virtually everything in the crypto section of the kernel configs, cbc fails to load. All the relevant modules are exist (dm-mod, dm-crypt, crypto_blkcipher, crypto_algapi, crypto_hash, aes_generic, sha256_generic), but even after modprobing / insmoding everything, /proc/crypto shows that aes and sha is there, but not cbc. The problem has been reproduced (using my kernel config) by Jonas Meurer, the Debian cryptsetup maintainer, so it's not just me ;). We've tried numerous different kernel versions in the .30 / .31 range. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=541835 Does this mean that something else somewhere in the kernel needs to be configured but isn't, and the necessary dependency isn't properly declared? My config is at: http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=45;filename=config-2.6.31-rc6-lizzie-00042-gb2add73;att=1;bug=541835 [I'm not subscribed to lkml; please cc me on responses] You could try/test a patch that was just posted: http://lkml.org/lkml/2009/8/27/249 that patch seems to be for ecryptfs only, while Celejar uses dm-crypt. second, the patch just ensures that CRYPTO_ECB and CRYPTO_CBC are selected along with ECRYPT_FS. but Celejar does have both CRYPTO_ECB and CRYPTO_CBC selected. ack. the problem rather is that loading the cbc blockcipher module simply does nothing. the module is listed in /proc/modules, but the blockcipher is still missing from /proc/crypto. the problem is not reproducible with a debian/unstable 2.6.30.6 kernel, even though it has cbc compiled as module as well. but if I recompile the same kernel sources with Celejars kernel .config, the problem occurs. thus it must be related to the kernel config in some way. add cc: linux-crypto@vger.kernel.org -- maybe someone there can help out. --- ~Randy *** Remember to use Documentation/SubmitChecklist when testing your code *** -- To unsubscribe from this list: send the line unsubscribe linux-crypto in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
