Jenkins build is back to normal : FreeBSD_HEAD #1796
See https://jenkins.freebsd.org/jenkins/job/FreeBSD_HEAD/1796/changes ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: pkg 1.4 freeze please test test test!
On Sat, 1 Nov 2014 23:45:49 +0100 Baptiste Daroussin b...@freebsd.org wrote: On Sat, Nov 01, 2014 at 04:13:32PM +0100, Marc UBM wrote: [snip] The update is failing for me with: .../usr/ports/ports-mgmt/pkg-devel# make all install clean === Installing for pkg-1.4.0.a3 === Checking if pkg already installed pkg-static: sqlite error while executing DROP INDEX packages_unique;CREATE UNIQUE INDEX packages_unique ON packages(name); in file pkgdb.c:2246: UNIQUE constraint failed: packages.name *** Error code 74 Stop. make[1]: stopped in /usr/ports/ports-mgmt/pkg-devel *** Error code 1 Stop. make: stopped in /usr/ports/ports-mgmt/pkg-devel portmaster fails with: root@ubm:/usr/ports/ports-mgmt/pkg-devel# portmaster -d pkg === No ORIGIN in /var/db/pkg/pkgconf-0.9.7/+CONTENTS === Cannot continue === Aborting update === Killing background jobs Terminated === Exiting make.conf related options: #enable pkgng (might be superfluous) WITH_PKGNG=yes #enable PKGNG devel WITH_PKGNG=devel Am I doing something wrong? You are doing nothing wrong but that probably means you have ancient packages that never got upgraded (in the old time it was allowed to have 2 packages installed with the same name) we have fixed that over the time and that is why we had unicity set to origin as a hack for a while, we are now moving to unique name so you have to make sure that all your installed packages are up to date before moving to new pkg. At least make sure you do not have 2 packages with the same name. Concerning portmaster I have no idea why it now thinks you are not using pkg :( I checked for packages with the same name and found none (via pkg version). Meanwhile, the upgrade still fails for me. Has anybody got any new ideas? Thanks in advance! Regards, Marc ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: How exactly does the base toolchain determine WHICH language to build with?
On 08 Nov 2014, at 01:23, Chris H bsd-li...@bsdforge.com wrote: Sorry for the long title. I've been [needlessly] struggling with getting ports within the ports tree to build, on a fresh 11-CURRENT install from 2014-11-05. With custom KERNEL and WORLD built, and installed. Here's my situation, which has worked well since ~8.2; make.conf(5) WITHOUT_CLANG=true FAVORITE_COMPILER=gcc src.conf(5) WITHOUT_CLANG=true If you don't want to use and build clang at all, I think you also need these two settings (at least on recent -CURRENT): WITHOUT_CLANG_BOOTSTRAP=foo WITHOUT_CLANG_IS_CC=foo The latter variable determines whether /usr/bin/cc, /usr/bin/c++ and so on are linked to clang or gcc. I suspect the root of all your problems is that you don't have this setting. I'll neither argue, nor defend rational for w/o clang. To boring and out of scope for this thread. That said; I realize that lang/clang(33/34/35) is the default toolchain for 10+, and that's just fine by me. So I shouldn't be terribly surprised when install kernel/world, followed by make delete-old removes the clang built, or provided by the base install from the (initial) install procedure. But what _does_ surprise me, is that the install of lang/gcc-48 does _not_ become the compiler of choice with the above $ENV, after [seemingly] deleting clang. It does not work that way. If you configure the base system to use gcc only, individual ports can decide that they need newer C++ support, or other features not available in the version of gcc in base (which is very old by now). In that case, they will use one of the gcc ports, or possibly even one of the clang ports. -Dimitry ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: CFR: AES-GCM and OpenCrypto work review
John-Mark Gurney j...@funkthat.com writes: Over the last few months, I've been working on a project to add support for AES-GCM and AES-CTR modes to our OpenCrypto framework. The work is sponsored by The FreeBSD Foundation and Netgate. I plan on committing these patches early next week. If you need more time for review, please email me privately and I will make delay. Please remember to bump __FreeBSD_version. DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: CFR: AES-GCM and OpenCrypto work review
On 08/11/14 04:23, John-Mark Gurney wrote: Hello, Over the last few months, I've been working on a project to add support for AES-GCM and AES-CTR modes to our OpenCrypto framework. The work is sponsored by The FreeBSD Foundation and Netgate. I plan on committing these patches early next week. If you need more time for review, please email me privately and I will make delay. The code has already been reviewed by Watson Ladd (the software crypto implementations) and Trevor Perrin (the aesni module part) and I have integrated these changes into the patch. There are two patches, one is the changes for OpenCrypto and the test framework. The other is the data files used by the test framework. The data is from NIST's CAVP program, and is about 20MB worth of test vectors. (I just realized, should we look at compressing these on disk?) Main patch (192KB): https://www.funkthat.com/~jmg/patches/aes.ipsec.5.patch Data files (~20MB): https://www.funkthat.com/~jmg/patches/aes.ipsec.5.testing.patch A list of notable changes in the patch: - Replacing crypto(4) w/ NetBSD's version + updates - Lots of man page updates, including CIOCFINDDEV and crypto(7) which adds specifics about restrictions on the modes. - Allow sane useage of both _HARDWARE and _SOFTWARE flags. - Add a timing safe bcmp for MAC comparision. - Add a software implementation of GCM that uses a four bit lookup table with parallelization. This algorithm is possibly vulnerable to timing attacks, but best known mitigation methods are used. Using a timing safe version is many times slower. - Added a CRYPTDEB macro that defaults to off. - Bring in some of OpenBSD's improvements to the OpenCrypto framework. - If an mbuf passed to the aesni module is only one segment, don't do a copy. This needs to be improved to support segmented buffers. - Remove the CRYPTO_F_REL flag. It was meaningless. It was used but did not change any behavior. - Add function crypto_mbuftoiov to convert an mbuf to an iov. This also converts the software crypto to only use iov's even for a simple linear buffer, and so simplifies the processing. - Add a dtrace probe for errors from the ioctl. - Add the CIOCCRYPTAEAD ioctl that allows userland processing (testing) of AES-GCM and future AEAD modes. Future improvements: - Support IV's longer than 12 bytes for GCM. - Make AES-NI support segmented buffers (iov or mbuf) so multisegmented inputs don't have to be copied. I have the question regarding to the algorithm of GF field calculations used in the proposed implementation: why not use the recent researches in GCM calculations, e.g. described in [1], for further speed optimizations? [1] - https://eprint.iacr.org/2013/157.pdf -- Vsevolod Stakhov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: CFR: AES-GCM and OpenCrypto work review
Vsevolod Stakhov wrote this message on Sat, Nov 08, 2014 at 18:55 +: On 08/11/14 04:23, John-Mark Gurney wrote: Hello, Over the last few months, I've been working on a project to add support for AES-GCM and AES-CTR modes to our OpenCrypto framework. The work is sponsored by The FreeBSD Foundation and Netgate. I plan on committing these patches early next week. If you need more time for review, please email me privately and I will make delay. The code has already been reviewed by Watson Ladd (the software crypto implementations) and Trevor Perrin (the aesni module part) and I have integrated these changes into the patch. There are two patches, one is the changes for OpenCrypto and the test framework. The other is the data files used by the test framework. The data is from NIST's CAVP program, and is about 20MB worth of test vectors. (I just realized, should we look at compressing these on disk?) Main patch (192KB): https://www.funkthat.com/~jmg/patches/aes.ipsec.5.patch Data files (~20MB): https://www.funkthat.com/~jmg/patches/aes.ipsec.5.testing.patch A list of notable changes in the patch: - Replacing crypto(4) w/ NetBSD's version + updates - Lots of man page updates, including CIOCFINDDEV and crypto(7) which adds specifics about restrictions on the modes. - Allow sane useage of both _HARDWARE and _SOFTWARE flags. - Add a timing safe bcmp for MAC comparision. - Add a software implementation of GCM that uses a four bit lookup table with parallelization. This algorithm is possibly vulnerable to timing attacks, but best known mitigation methods are used. Using a timing safe version is many times slower. - Added a CRYPTDEB macro that defaults to off. - Bring in some of OpenBSD's improvements to the OpenCrypto framework. - If an mbuf passed to the aesni module is only one segment, don't do a copy. This needs to be improved to support segmented buffers. - Remove the CRYPTO_F_REL flag. It was meaningless. It was used but did not change any behavior. - Add function crypto_mbuftoiov to convert an mbuf to an iov. This also converts the software crypto to only use iov's even for a simple linear buffer, and so simplifies the processing. - Add a dtrace probe for errors from the ioctl. - Add the CIOCCRYPTAEAD ioctl that allows userland processing (testing) of AES-GCM and future AEAD modes. Future improvements: - Support IV's longer than 12 bytes for GCM. - Make AES-NI support segmented buffers (iov or mbuf) so multisegmented inputs don't have to be copied. I have the question regarding to the algorithm of GF field calculations used in the proposed implementation: why not use the recent researches in GCM calculations, e.g. described in [1], for further speed optimizations? [1] - https://eprint.iacr.org/2013/157.pdf The paper you linked to does not describe a new way of calculating GHASH, but evalutation of a bug in their implementation using the PCLMULQDQ instruction... If you mean, why don't I use OpenSSL's code? The reason is that their code is a perl script that generates assmebly... We don't have perl in base.. and I didn't want more assembly in our tree (see below).. Instead, I decided to use code from Intel's whitepaper: Intel® Carry-Less Multiplication Instruction and its Usage for Computing the GCM Mode I didn't use their assembly version because I wanted to have maintainable code, and also the same code can be used on both i386 and amd64 arches.. This turns out to also be a good thing, as when I add segmented buffer support, it'll be much easier to add to the C version, and I only have to do the work once for two arches... Also, the software GF library that I wrote is using state of the art algorithms... An OpenBSD developer has tested my code and has seen a significant performance improvement over their old code, and are evaluating if they want to/can include it in their tree... Hope this answers your question. If not, please be more specific so I can answer it. Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 All that I will do, has been done, All that I have, has not. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: How exactly does the base toolchain determine WHICH language to build with?
On Sat, 8 Nov 2014 14:46:21 +0100 Dimitry Andric d...@freebsd.org wrote On 08 Nov 2014, at 01:23, Chris H bsd-li...@bsdforge.com wrote: Sorry for the long title. I've been [needlessly] struggling with getting ports within the ports tree to build, on a fresh 11-CURRENT install from 2014-11-05. With custom KERNEL and WORLD built, and installed. Here's my situation, which has worked well since ~8.2; make.conf(5) WITHOUT_CLANG=true FAVORITE_COMPILER=gcc src.conf(5) WITHOUT_CLANG=true If you don't want to use and build clang at all, I think you also need these two settings (at least on recent -CURRENT): WITHOUT_CLANG_BOOTSTRAP=foo WITHOUT_CLANG_IS_CC=foo The latter variable determines whether /usr/bin/cc, /usr/bin/c++ and so on are linked to clang or gcc. I suspect the root of all your problems is that you don't have this setting. I'll neither argue, nor defend rational for w/o clang. To boring and out of scope for this thread. That said; I realize that lang/clang(33/34/35) is the default toolchain for 10+, and that's just fine by me. So I shouldn't be terribly surprised when install kernel/world, followed by make delete-old removes the clang built, or provided by the base install from the (initial) install procedure. But what _does_ surprise me, is that the install of lang/gcc-48 does _not_ become the compiler of choice with the above $ENV, after [seemingly] deleting clang. It does not work that way. If you configure the base system to use gcc only, individual ports can decide that they need newer C++ support, or other features not available in the version of gcc in base (which is very old by now). In that case, they will use one of the gcc ports, or possibly even one of the clang ports. Ahh. Thank you. It's now perfectly clear, and makes complete sense, now. For the record; I have nothing against clang/llvm, per se. I am just looking to obtain a bit more control over my environment. As this installation is intended for development purposes. In fact, I'll probably install all 3 versions of lang/clang. :) Oh, and GNC C, as well. Thank you again, for the reply. Now that I have also been informed about freebsd-toolchain@. I should probably take this opportunity to apologize for the noise, and thank all for their thoughtful replies. :) --Chris -Dimitry ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: CFR: AES-GCM and OpenCrypto work review
On 08/11/14 20:45, John-Mark Gurney wrote: Vsevolod Stakhov wrote this message on Sat, Nov 08, 2014 at 18:55 +: On 08/11/14 04:23, John-Mark Gurney wrote: Hello, Over the last few months, I've been working on a project to add support for AES-GCM and AES-CTR modes to our OpenCrypto framework. The work is sponsored by The FreeBSD Foundation and Netgate. I plan on committing these patches early next week. If you need more time for review, please email me privately and I will make delay. The code has already been reviewed by Watson Ladd (the software crypto implementations) and Trevor Perrin (the aesni module part) and I have integrated these changes into the patch. There are two patches, one is the changes for OpenCrypto and the test framework. The other is the data files used by the test framework. The data is from NIST's CAVP program, and is about 20MB worth of test vectors. (I just realized, should we look at compressing these on disk?) Main patch (192KB): https://www.funkthat.com/~jmg/patches/aes.ipsec.5.patch Data files (~20MB): https://www.funkthat.com/~jmg/patches/aes.ipsec.5.testing.patch A list of notable changes in the patch: - Replacing crypto(4) w/ NetBSD's version + updates - Lots of man page updates, including CIOCFINDDEV and crypto(7) which adds specifics about restrictions on the modes. - Allow sane useage of both _HARDWARE and _SOFTWARE flags. - Add a timing safe bcmp for MAC comparision. - Add a software implementation of GCM that uses a four bit lookup table with parallelization. This algorithm is possibly vulnerable to timing attacks, but best known mitigation methods are used. Using a timing safe version is many times slower. - Added a CRYPTDEB macro that defaults to off. - Bring in some of OpenBSD's improvements to the OpenCrypto framework. - If an mbuf passed to the aesni module is only one segment, don't do a copy. This needs to be improved to support segmented buffers. - Remove the CRYPTO_F_REL flag. It was meaningless. It was used but did not change any behavior. - Add function crypto_mbuftoiov to convert an mbuf to an iov. This also converts the software crypto to only use iov's even for a simple linear buffer, and so simplifies the processing. - Add a dtrace probe for errors from the ioctl. - Add the CIOCCRYPTAEAD ioctl that allows userland processing (testing) of AES-GCM and future AEAD modes. Future improvements: - Support IV's longer than 12 bytes for GCM. - Make AES-NI support segmented buffers (iov or mbuf) so multisegmented inputs don't have to be copied. I have the question regarding to the algorithm of GF field calculations used in the proposed implementation: why not use the recent researches in GCM calculations, e.g. described in [1], for further speed optimizations? [1] - https://eprint.iacr.org/2013/157.pdf The paper you linked to does not describe a new way of calculating GHASH, but evalutation of a bug in their implementation using the PCLMULQDQ instruction... If you mean, why don't I use OpenSSL's code? The reason is that their code is a perl script that generates assmebly... We don't have perl in base.. and I didn't want more assembly in our tree (see below).. Instead, I decided to use code from Intel's whitepaper: Intel® Carry-Less Multiplication Instruction and its Usage for Computing the GCM Mode I didn't use their assembly version because I wanted to have maintainable code, and also the same code can be used on both i386 and amd64 arches.. This turns out to also be a good thing, as when I add segmented buffer support, it'll be much easier to add to the C version, and I only have to do the work once for two arches... Also, the software GF library that I wrote is using state of the art algorithms... An OpenBSD developer has tested my code and has seen a significant performance improvement over their old code, and are evaluating if they want to/can include it in their tree... Hope this answers your question. If not, please be more specific so I can answer it. Thanks. I'm sorry, I thought that is the paper that is a transcript of the following presentation: http://2013.diac.cr.yp.to/slides/gueron.pdf made by the same authors. The transcript is not available so far it seems. And regarding assembler/C maintainability I would argue that the current intrinsics based implementation is more readable than the pure assembler solution (and it is still machine dependent). Of course, I'm not the expert in such optimizations, so that is just my own feeling. By the way, do you have some concrete numbers about the performance of your aes-gcm? (I recently could do aes-128-gcm at about 32 gigabits/sec that is not a limit of the modern hardware for sure). ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
