Re: [Cryptodev-linux-devel] OpenSSH/OpenSSL with Cryptodev

[Gordan Bobic]

To answer my own question at last:
The problem was a RHEL specific audit related patch to OpenSSH.
The said audit patch is quite large, intrusive, poorly tested
to even work with the configuration options RH broke out
as build-time configurable, and breaks cryptodev offload.
The solution was to back out that patch, and fix all the
subsequent patches that broke to resolve the ensuing FTBFS.

Patched EL6 source packages for OpenSSL (-DHAVE_CRYPTODEV) and
OpenSSH (with audit patch backed out) can be acquired from here:

http://ftp.redsleeve.org/pub/el6/packages/soc/SRPMS/openssl-1.0.1e-30.el6.11.cryptodev.src.rpm
http://ftp.redsleeve.org/pub/el6/packages/soc/SRPMS/openssh-5.3p1-104.el6.1.cryptodev.src.rpm

I hope this saves others time in the future.

Gordan

On 2015-06-24 13:50, Gordan Bobic wrote:

I've been trying to get this working for a while now and I am having
absolutely no luck. I just built vanilla OpenSSL 1.0.1o, and I still
get the same problem when an offloadable crypto algorithm is used
(note - not specific to hardware acceleration, the same thing
happens if the crypto is any in-kernel software implementation):

sshd: fatal: evp_crypt: EVP_Cipher failed

Has anybody got this working with any combination of OpenSSH and
OpenSSL versions?

For completeness, mod_ssl and openssl pipes using s_client and
s_server work offloaded as expected, it is only sshd that seems
to fail when an offloadable algorithm is used, but unless I'm
misreading the code the above error arises due to an error
return from OpenSSL rather than within OpenSSH.

If anybody has a version combination that works I would really
like to know.

TIA.

Gordan

___
Cryptodev-linux-devel mailing list
Cryptodev-linux-devel@gna.org
https://mail.gna.org/listinfo/cryptodev-linux-devel



___
Cryptodev-linux-devel mailing list
Cryptodev-linux-devel@gna.org
https://mail.gna.org/listinfo/cryptodev-linux-devel

[Cryptodev-linux-devel] OpenSSH/OpenSSL with Cryptodev

[Gordan Bobic]

I've been trying to get this working for a while now and I am having
absolutely no luck. I just built vanilla OpenSSL 1.0.1o, and I still
get the same problem when an offloadable crypto algorithm is used
(note - not specific to hardware acceleration, the same thing
happens if the crypto is any in-kernel software implementation):

sshd: fatal: evp_crypt: EVP_Cipher failed

Has anybody got this working with any combination of OpenSSH and
OpenSSL versions?

For completeness, mod_ssl and openssl pipes using s_client and
s_server work offloaded as expected, it is only sshd that seems
to fail when an offloadable algorithm is used, but unless I'm
misreading the code the above error arises due to an error
return from OpenSSL rather than within OpenSSH.

If anybody has a version combination that works I would really
like to know.

TIA.

Gordan

___
Cryptodev-linux-devel mailing list
Cryptodev-linux-devel@gna.org
https://mail.gna.org/listinfo/cryptodev-linux-devel

Re: [Cryptodev-linux-devel] cryptodev, mv_cesa & sha1

[Gordan Bobic]

Apologies if this is stated earlier in the thread, but you did build
your OpenSSL with -DUSE_CRYPTODEV_DIGESTS, right?

What version of OpenSSL are you using? Have you perchance checked
whether sshd works with cryptodev offload (ssh -c aes128-cbc to your
cryptodev machine)?

Gordan

On 30/05/15 12:43, JM wrote:

Hello again,

I have updated the kernel on the affected machine to 4.0.4 and now
mv_cesa successfully passes self checks. I rebuilt the cryptodev-linux
module with the compilation fix from git and the tests included in
cryptodev also pass now.

However, I am having trouble doing sha1 hashing using mv_cesa through
cryptodev from other applications, while aes-128-cbc works correctly.
Tested with openssl (built with the recommended patches and flags):

AES:
openssl speed aes-128-cbc -elapsed
software path is taken

openssl speed -evp aes-128-cbc -elapsed
offloading works correctly, and I see mv_crypto taking CPU time

SHA1:
openssl speed sha1 -elapsed
software path is taken

openssl speed -evp sha1 -elapsed
software path is taken, no difference in speed, mv_crypto at 0% CPU time

this is what I get in dmesg with verbosity set to 3:
-evp aes: http://pastebin.com/K3TaJ2S8
-evp sha1: http://pastebin.com/udK6EYwS

In /proc/crypto I have

name : sha1
driver   : mv-sha1
module   : mv_cesa
priority : 300
refcnt   : 1
selftest : passed
type : ahash
async: yes
blocksize: 64
digestsize   : 20

which is the highest priority for sha1 that I have in this system.

Importantly, ./sha_speed from cryptodev-linux/tests correctly requests
mv-sha1 and I can see it use mv_crypto in the process list with about
45% cpu usage.

To summarize, openssl can use mv_cesa through cryptodev only for
aes-128-cbc and not for sha1, even though tests included in cryptodev
can use mv_cesa for both aes and sha1.

Can someone shed any light on that?

Best regards,
Jan


On Mon, Apr 27, 2015 at 2:24 PM, Phil Sutter mailto:p...@nwl.cc>> wrote:

Hi,

On Mon, Apr 27, 2015 at 02:08:29PM +0200, JM wrote:
> Hello,
>
> Thank you for the quick reply.
>
> I use the mv_cesa driver as shipped by Debian with the 3.16 kernel, which
> is almost certainly the same as mainline.
>
> I tried to run the _comp tools with mv_cesa removed:
>
> fijam@yukikaze:~/cryptodev-linux-1.7/tests$ ./cipher_comp && ./hash_comp
> requested cipher CRYPTO_AES_CBC and mac CRYPTO_SHA1_HMAC, got cipher
> cbc(aes) with driver cbc(aes-generic) and hash  with driver
> requested mac CRYPTO_SHA1, got hash sha1 with driver sha1-asm
>
> and loaded:
>
> fijam@yukikaze:~/cryptodev-linux-1.7/tests$ sudo modprobe mv_cesa
> fijam@yukikaze:~/cryptodev-linux-1.7/tests$ ./cipher_comp && ./hash_comp
> requested cipher CRYPTO_AES_CBC and mac CRYPTO_SHA1_HMAC, got cipher
> cbc(aes) with driver mv-cbc-aes and hash  with driver
> requested mac CRYPTO_SHA1, got hash sha1 with driver sha1-asm
>
> They both pass, but note that even with mv_cesa loaded, sha1-asm is used.
>
> I checked /proc/crypto and mv_cesa has indeed the highest priority (300). 
I
> tried to rmmod sha1_generic and sha1_arm but they just get loaded again.

This wouldn't work anyway since mv_cesa needs a software implementation
to fall back to for odd cases.

> I am not very familiar with kernel self-checks. I tested with the tcrypt
> module (mode 200 for aes and mode 303 for sha) and I think I'm on the 
right
> track:
>
> [227229.582143] alg: hash: Test 3 failed for mv-sha1
> [227229.586891] : 10 bf d7 00 71 0b bb 83 3a 26 d0 97 13 05 99 f5
> [227229.593454] 0010: 3a 92 53 3c
> [227229.597165] alg: hash: Test 1 failed for mv-hmac-sha1
> [227229.602343] : 0c aa 9f d5 37 c3 79 3a 91 d9 21 5f 42 2b 2c 24
> [227229.608906] 0010: b7 c3 16 0c
> [227251.303414]
> [227251.303414] testing speed of sha1
> [227251.323114] test  0 (   16 byte blocks,   16 bytes per update,   1
> updates): 518331 opers/sec,   8293296 bytes/sec
> [and so on]
>
> It would appear that mv_sha1 fails, at which point the generic driver gets
> loaded?
>
> Does it mean the kernel driver itself is at fault?

Yes, indeed. I'm surprised it still turns up in /proc/crypto, but if the
kernel self-test for a given crypto-provider fails it won't be used
anymore (for good reason, of course).

The driver failing with tcrypt is clear evidence of a driver problem and
unrelated to cryptodev. Have a look at the kernel log, I bet there is
output from the failing self-test at boot-time.

Cheers, Phil

___
Cryptodev-linux-devel mailing list
Cryptodev-linux-devel@gna.org 
https://mail.gna.org/listinfo/cryptodev-linux-devel




___
Cryptodev-linux-devel ma

Re: [Cryptodev-linux-devel] Problem with OpenSSH/OpenSSL Interaction When Cryptodev is Used

[Gordan Bobic]

On 2015-05-28 14:26, Gordan Bobic wrote:

On 2015-05-28 13:12, Nikos Mavrogiannopoulos wrote:
On Thu, May 28, 2015 at 12:06 PM, Gordan Bobic  
wrote:

Does that mean it is not a 0-copy related problem?

In fact, I can confirm the problem is not at all related to mv_cesa.
I just unloaded the mv_cesa driver which means that with cryptodev
it will be the kernel doing the AES using the generic kernel
driver. And ssh still fails with that when an offloadable algorithm
is being used. So the problem is almost certainly somewhere in the
OpenSSL <-> cryptodev interaction.


The openssl cryptodev support is broken and that is mentioned in the
readme of cryptodev.
It requires applying at least the patches from:
https://github.com/openssl/openssl/pull/191

and I have no idea whether that would be sufficient in their current
releases. I haven't followed up for quite some time.


Thanks for the hint. I have adapted the patches for the version of
OpenSSL that ships with my distro (EL6). I have kicked off the
rebuild so will know in a few hours whether it worked.

Thanks for your input, most appreciated. :)


Update:

Rebuilt OpenSSL with those two patches - no difference.
sshd still fails, everything else still works.

Gordan

___
Cryptodev-linux-devel mailing list
Cryptodev-linux-devel@gna.org
https://mail.gna.org/listinfo/cryptodev-linux-devel

Re: [Cryptodev-linux-devel] Problem with OpenSSH/OpenSSL Interaction When Cryptodev is Used

[Gordan Bobic]

On 2015-05-28 13:12, Nikos Mavrogiannopoulos wrote:
On Thu, May 28, 2015 at 12:06 PM, Gordan Bobic  
wrote:

Does that mean it is not a 0-copy related problem?

In fact, I can confirm the problem is not at all related to mv_cesa.
I just unloaded the mv_cesa driver which means that with cryptodev
it will be the kernel doing the AES using the generic kernel
driver. And ssh still fails with that when an offloadable algorithm
is being used. So the problem is almost certainly somewhere in the
OpenSSL <-> cryptodev interaction.


The openssl cryptodev support is broken and that is mentioned in the
readme of cryptodev.
It requires applying at least the patches from:
https://github.com/openssl/openssl/pull/191

and I have no idea whether that would be sufficient in their current
releases. I haven't followed up for quite some time.


Thanks for the hint. I have adapted the patches for the version of
OpenSSL that ships with my distro (EL6). I have kicked off the
rebuild so will know in a few hours whether it worked.

Thanks for your input, most appreciated. :)

Gordan

___
Cryptodev-linux-devel mailing list
Cryptodev-linux-devel@gna.org
https://mail.gna.org/listinfo/cryptodev-linux-devel

Re: [Cryptodev-linux-devel] Problem with OpenSSH/OpenSSL Interaction When Cryptodev is Used

[Gordan Bobic]

On 2015-05-28 11:00, Gordan Bobic wrote:

On 2015-05-27 23:33, Phil Sutter wrote:

On Wed, May 27, 2015 at 10:36:29PM +0100, Gordan Bobic wrote:

What do I need to change in which file to disable 0-copy?


Have a look at main.c, line 239:

| if (cop->flags & COP_FLAG_NO_ZC)
| ret = __crypto_run_std(ses_ptr, &kcop->cop);
| else
| ret = __crypto_run_zc(ses_ptr, kcop);

Changing the first line to 'if (1)' should suffice.


I did that and the problem persists:

# ./hmac_comp
requested cipher CRYPTO_AES_CBC and mac CRYPTO_SHA1_HMAC, got cipher
cbc(aes) with driver mv-cbc-aes and hash hmac(sha1) with driver
mv-hmac-sha1
fail for datalen 0x10, MACs do not match!
wrong mac:
\xd7\xd1\xa6\xef\x0a\x38\xe1\x09\x45\xe1\x8b\x48\x88\xaa\xa9\x23\x4c\xd4\x67\xd1
right mac:
\xd7\xd1\xa6\xef\x0a\x38\xe1\x09\x45\xe1\x8b\x48\x88\xaa\xa9\x23\x4c\xd4\x67\xd1
test_crypto() failed for datalen of 16

And ssh still fails with the same symptoms when using an
offloadable algorithm with cryptodev disabled.

Does that mean it is not a 0-copy related problem?


In fact, I can confirm the problem is not at all related to mv_cesa.
I just unloaded the mv_cesa driver which means that with cryptodev
it will be the kernel doing the AES using the generic kernel
driver. And ssh still fails with that when an offloadable algorithm
is being used. So the problem is almost certainly somewhere in the
OpenSSL <-> cryptodev interaction.

Gordan

___
Cryptodev-linux-devel mailing list
Cryptodev-linux-devel@gna.org
https://mail.gna.org/listinfo/cryptodev-linux-devel

Re: [Cryptodev-linux-devel] Problem with OpenSSH/OpenSSL Interaction When Cryptodev is Used

[Gordan Bobic]

On 2015-05-27 23:33, Phil Sutter wrote:

On Wed, May 27, 2015 at 10:36:29PM +0100, Gordan Bobic wrote:

What do I need to change in which file to disable 0-copy?


Have a look at main.c, line 239:

| if (cop->flags & COP_FLAG_NO_ZC)
| ret = __crypto_run_std(ses_ptr, &kcop->cop);
| else
| ret = __crypto_run_zc(ses_ptr, kcop);

Changing the first line to 'if (1)' should suffice.


I did that and the problem persists:

# ./hmac_comp
requested cipher CRYPTO_AES_CBC and mac CRYPTO_SHA1_HMAC, got cipher 
cbc(aes) with driver mv-cbc-aes and hash hmac(sha1) with driver 
mv-hmac-sha1

fail for datalen 0x10, MACs do not match!
wrong mac: 
\xd7\xd1\xa6\xef\x0a\x38\xe1\x09\x45\xe1\x8b\x48\x88\xaa\xa9\x23\x4c\xd4\x67\xd1
right mac: 
\xd7\xd1\xa6\xef\x0a\x38\xe1\x09\x45\xe1\x8b\x48\x88\xaa\xa9\x23\x4c\xd4\x67\xd1

test_crypto() failed for datalen of 16

And ssh still fails with the same symptoms when using an
offloadable algorithm with cryptodev disabled.

Does that mean it is not a 0-copy related problem?

Gordan

___
Cryptodev-linux-devel mailing list
Cryptodev-linux-devel@gna.org
https://mail.gna.org/listinfo/cryptodev-linux-devel

Re: [Cryptodev-linux-devel] Problem with OpenSSH/OpenSSL Interaction When Cryptodev is Used

[Gordan Bobic]

On 2015-05-27 20:34, Phil Sutter wrote:

On Wed, May 27, 2015 at 11:20:28AM +0100, Gordan Bobic wrote:

On 2015-05-27 11:07, Phil Sutter wrote:
> Hi,
>
> On Wed, May 27, 2015 at 10:25:00AM +0100, Gordan Bobic wrote:
>> ./cipher-aead-srtp
>> ioctl(CIOCGSESSION): Invalid argument
>
> Not sure if this should succeed at all, at least that's not my code. ;)
>
>> # ./hmac_comp
>> requested cipher CRYPTO_AES_CBC and mac CRYPTO_SHA1_HMAC, got cipher
>> cbc(aes) with driver mv-cbc-aes and hash hmac(sha1) with driver
>> mv-hmac-sha1
>> fail for datalen 0x10, MACs do not match!
>> wrong mac:
>> 
\xd7\xd1\xa6\xef\x0a\x38\xe1\x09\x45\xe1\x8b\x48\x88\xaa\xa9\x23\x4c\xd4\x67\xd1
>> right mac:
>> 
\xd7\xd1\xa6\xef\x0a\x38\xe1\x09\x45\xe1\x8b\x48\x88\xaa\xa9\x23\x4c\xd4\x67\xd1
>> test_crypto() failed for datalen of 16
>
> That's bad. I'm pretty sure SSH uses HMAC. Unless there is a bug in
> hmac_comp, the fact that identical binary strings are shown for actual
> and expected result makes me suspect a caching issue (the additional
> data access while printing sometimes leads to a cache flush so one does
> not see the complained difference).

But - my OpenSSL is built without -DUSE_CRYPTODEV_DIGESTS, so if 
OpenSSH

used OpenSSL for that, there should be no attempt to even use hardware
digests.


Does this cover HMAC as well?


I assumed so. It hadn't occurred to me it might not.


> You are using MV_CESA, therefore you are probably on a Kirkwood with
> VIVT cache architecture - which is pretty delicate when it comes to the
> zero-copy stuff involved. Please make sure your kernel doesn't miss any
> fixes in mv_cesa regarding cache flushes.

That is easier said than done. I'm using the upstream vendor provided
sources that are both ancient (3.4.6) and heavily patched (the diff
between mainline 3.4.6 and their kernel is MBs).


Sounds familiar. For mv_cesa though, it should be possible to just take
over mainline mv_cesa.c with minimal adjustments (drop clock gating 
e.g.).


I'll see if I can get the latest mv_cesa module to build/work.


> Also, for the sake of testing
> you could turn off zero-copy in cryptodev as well.

How? I don't see any obvious options in the Makefile to do that.
All I see is the async option, which is disabled by default.


You can set COP_FLAG_NO_ZC in struct crypt_op. So you need to recompile
openssl. Alternatively you could change the code in cryptodev referring
to that flag so it always behaves as if it had been set.


Changing the cryptodev code sounds like the way forward for the test
purposes at least, mainly because rebuilding cryptodev takes a minute
or so, and rebuilding the OpenSSL package takes a couple of hours.

What do I need to change in which file to disable 0-copy?

Gordan

___
Cryptodev-linux-devel mailing list
Cryptodev-linux-devel@gna.org
https://mail.gna.org/listinfo/cryptodev-linux-devel

Re: [Cryptodev-linux-devel] Problem with OpenSSH/OpenSSL Interaction When Cryptodev is Used

[Gordan Bobic]

On 2015-05-27 11:07, Phil Sutter wrote:

Hi,

On Wed, May 27, 2015 at 10:25:00AM +0100, Gordan Bobic wrote:

./cipher-aead-srtp
ioctl(CIOCGSESSION): Invalid argument


Not sure if this should succeed at all, at least that's not my code. ;)


# ./hmac_comp
requested cipher CRYPTO_AES_CBC and mac CRYPTO_SHA1_HMAC, got cipher
cbc(aes) with driver mv-cbc-aes and hash hmac(sha1) with driver
mv-hmac-sha1
fail for datalen 0x10, MACs do not match!
wrong mac:
\xd7\xd1\xa6\xef\x0a\x38\xe1\x09\x45\xe1\x8b\x48\x88\xaa\xa9\x23\x4c\xd4\x67\xd1
right mac:
\xd7\xd1\xa6\xef\x0a\x38\xe1\x09\x45\xe1\x8b\x48\x88\xaa\xa9\x23\x4c\xd4\x67\xd1
test_crypto() failed for datalen of 16


That's bad. I'm pretty sure SSH uses HMAC. Unless there is a bug in
hmac_comp, the fact that identical binary strings are shown for actual
and expected result makes me suspect a caching issue (the additional
data access while printing sometimes leads to a cache flush so one does
not see the complained difference).


But - my OpenSSL is built without -DUSE_CRYPTODEV_DIGESTS, so if OpenSSH
used OpenSSL for that, there should be no attempt to even use hardware
digests.


You are using MV_CESA, therefore you are probably on a Kirkwood with
VIVT cache architecture - which is pretty delicate when it comes to the
zero-copy stuff involved. Please make sure your kernel doesn't miss any
fixes in mv_cesa regarding cache flushes.


That is easier said than done. I'm using the upstream vendor provided
sources that are both ancient (3.4.6) and heavily patched (the diff
between mainline 3.4.6 and their kernel is MBs).


Also, for the sake of testing
you could turn off zero-copy in cryptodev as well.


How? I don't see any obvious options in the Makefile to do that.
All I see is the async option, which is disabled by default.

Gordan

___
Cryptodev-linux-devel mailing list
Cryptodev-linux-devel@gna.org
https://mail.gna.org/listinfo/cryptodev-linux-devel

Re: [Cryptodev-linux-devel] Problem with OpenSSH/OpenSSL Interaction When Cryptodev is Used

[Gordan Bobic]

On 2015-05-27 01:08, Phil Sutter wrote:

Hi,

On Tue, May 26, 2015 at 07:29:34PM +0100, Gordan Bobic wrote:

A bit of extra info, with cryptodev_verbosity=2, on 0.9 when the error
occurs:
cryptodev: sshd[1205]: invalid session ID=0xAADBA6A0

With 1.7:
cryptodev: sshd[1520] (fill_kcop_from_cop:647): invalid session
ID=0xEC91F39A
cryptodev: sshd[1520] (cryptodev_ioctl:857): Error copying from user


But the test code (i.e. the various *_comp programs) succeeds?


Thank you for that hint:
# cd tests
# make
./cipher
./hmac
./async_cipher
./async_hmac
./cipher-aead-srtp
ioctl(CIOCGSESSION): Invalid argument
make: *** [check] Error 1

In dmesg:
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
80f553e0
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
a2bd0180
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
a2bd06c0
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
a2bd0e40
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
a2bd0780
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
a2bd0b40
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
a2bd03c0
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
b4db8b00
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
b4db8ec0
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
b4db82c0
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
b4db8a40
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
a2bd0840
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
a2bd0600
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
a2bd0480
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
a2bd0240
cryptodev: cipher[13263] (cryptodev_open:480): allocated new item at 
a2bd0c00
cryptodev: cipher[13263] (cryptodev_open:485): Cryptodev handle 
initialised, 16 elements in queue

cryptodev: cipher[13263] (crypto_create_session:284): got alignmask 0
cryptodev: cipher[13263] (crypto_create_session:287): preallocating for 
32 user pages

cryptodev: cipher[13263] (crypto_create_session:284): got alignmask 0
cryptodev: cipher[13263] (crypto_create_session:287): preallocating for 
32 user pages
cryptodev: cipher[13263] (crypto_destroy_session:341): Removed session 
0xB286F3D7
cryptodev: cipher[13263] (crypto_destroy_session:344): freeing space for 
32 user pages

cryptodev: cipher[13263] (crypto_create_session:284): got alignmask 0
cryptodev: cipher[13263] (crypto_create_session:287): preallocating for 
32 user pages
cryptodev: cipher[13263] (crypto_destroy_session:341): Removed session 
0x781F7717
cryptodev: cipher[13263] (crypto_destroy_session:344): freeing space for 
32 user pages

cryptodev: cipher[13263] (crypto_create_session:284): got alignmask 0
cryptodev: cipher[13263] (crypto_create_session:287): preallocating for 
32 user pages
cryptodev: cipher[13263] (crypto_destroy_session:341): Removed session 
0x89FD4E64
cryptodev: cipher[13263] (crypto_destroy_session:344): freeing space for 
32 user pages
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
a2bd0c00
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
a2bd0240
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
a2bd0480
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
a2bd0600
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
a2bd0840
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
b4db8a40
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
b4db82c0
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
b4db8ec0
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
b4db8b00
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
a2bd03c0
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
a2bd0b40
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
a2bd0780
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
a2bd0e40
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
a2bd06c0
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
a2bd0180
cryptodev: cipher[13263] (cryptodev_release:519): freeing item at 
80f553e0
cryptodev: cipher[13263] (crypto_destroy_session:341): Removed session 
0x8AD52BF7
cryptodev: cipher[13263] (crypto_destroy_session:344): freeing space for 
32 user pages
cryptodev: cipher[13263] (cryptodev_release:541): Cryptodev handle 
deinitialised, 16 elements freed
cryptodev: hmac[13264] (cryptodev_open:480): allocated new item at 
a2bd09c0
cryptodev: hmac[13264] (cryptodev_open:480): allocated new item at 
a2bd00c0
cryptodev: hmac[13264] (cryptodev_open:480): allocated new item at 
80f553e0
cryptodev: hmac[13264] (cryptodev_open:480): allocated new item at 
a2bd0180
cryptodev: hmac[13264] (cryptodev_open:480

Re: [Cryptodev-linux-devel] Problem with OpenSSH/OpenSSL Interaction When Cryptodev is Used

[Gordan Bobic]

On 2015-05-26 17:05, Gordan Bobic wrote:
I am experiencing a rather odd problem with cryptodev that I am not 
sure

where to follow next.

The distro is RedSleeve (EL6 armv5tel).
OpenSSH 5.3p1 (EL6 extra patches)
OpenSSL 1.0.1e (EL6 extra patches), rebuild with -DHAVE_CRYPTODEV
(without digests)
Cryptodev 0.9 (yes, I know this is ancient but I tried 1.7 and it made
no difference)
Linux 3.4.6 (old, but heavily patched for both the hardware
compatibility and CVEs, binary that ships with my device)

What works:
mod_ssl with ECDHE-RSA-AES128-GCM-SHA256
openssl s_server <-> s_client

What doesn't:
sshd with offloadable ciphers
Non-offloadable ciphers (e.g. blowfish) always work fine.
aes128-cbc works without cryptodev module loaded, but when it is
loaded the connection breaks _after_ sending the last login
information.


With cryptodev loaded and Ciphers set to aes128-cbc:
On the client side, everything seems to work fine, right up to the end:

$ ssh localhost
[...]
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Last login: Tue May 26 16:40:17 2015 from localhost
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 client-session (t4 r0 i0/0 o0/0 fd 4/5 cc -1)

Connection to qnap closed by remote host.
Connection to qnap closed.
Transferred: sent 3472, received 4128 bytes, in 0.5 seconds
Bytes per second: sent 7622.1, received 9062.3
debug1: Exit status -1

This tells me that the connection setup actually succeeds, since this
gets received by the client:
Last login: Tue May 26 16:40:17 2015 from localhost

On the server side with LogLevel DEBUG:
Everything looks OK initially:
[...]
May 26 16:59:49 qnap sshd[29520]: debug1: Allocating pty.
May 26 16:59:49 qnap sshd[29474]: debug1: session_new: session 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_pty_req: session 0
alloc /dev/pts/4
May 26 16:59:49 qnap sshd[29520]: debug1: server_input_channel_req:
channel 0 request env reply 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_by_channel: session
0 channel 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_input_channel_req:
session 0 req env
May 26 16:59:49 qnap sshd[29520]: debug1: server_input_channel_req:
channel 0 request env reply 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_by_channel: session
0 channel 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_input_channel_req:
session 0 req env
May 26 16:59:49 qnap sshd[29520]: debug1: server_input_channel_req:
channel 0 request shell reply 1
May 26 16:59:49 qnap sshd[29520]: debug1: session_by_channel: session
0 channel 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_input_channel_req:
session 0 req shell
May 26 16:59:49 qnap sshd[29521]: debug1: Setting controlling tty
using TIOCSCTTY.

Right up to here:

May 26 16:59:49 qnap sshd[29520]: fatal: evp_crypt: EVP_Cipher failed

The rest appears to be just clean-up:

May 26 16:59:49 qnap sshd[29520]: debug1: do_cleanup
May 26 16:59:49 qnap sshd[29474]: debug1: do_cleanup
May 26 16:59:49 qnap sshd[29474]: debug1: PAM: cleanup
May 26 16:59:49 qnap sshd[29474]: debug1: PAM: closing session
May 26 16:59:49 qnap sshd[29474]: pam_unix(sshd:session): session
closed for user gordan
May 26 16:59:49 qnap sshd[29474]: debug1: PAM: deleting credentials
May 26 16:59:49 qnap sshd[29474]: debug1: session_pty_cleanup: session
0 release /dev/pts/4


The error:
sshd[29520]: fatal: evp_crypt: EVP_Cipher failed
appears to be emitted by sshd rather than OpenSSL or
Cryptodev. The relevant bit of code is in OpenSSH's
cipher.c file:

void
cipher_crypt(CipherContext *cc, u_char *dest, const u_char *src, u_int
len)
{
 if (len % cc->cipher->block_size)
 fatal("cipher_encrypt: bad plaintext length %d", len);
 if (EVP_Cipher(&cc->evp, dest, (u_char *)src, len) == 0)
 fatal("evp_crypt: EVP_Cipher failed");
}

EVP_Cipher function is part of OpenSSL, declared in:
crypto/evp/evp.h:
int EVP_Cipher(EVP_CIPHER_CTX *c,
 unsigned char *out,
 const unsigned char *in,
 unsigned int inl);

and defined in:
crypto/evp/evp_lib.c:
int EVP_Cipher(EVP_CIPHER_CTX *ctx, unsigned char *out, const unsigned
char *in, unsigned int inl)
 {
#ifdef OPENSSL_FIPS
 FIPS_selftest_check();
#endif
 return ctx->cipher->do_cipher(ctx,out,in,inl);
 }

So it would appear that
ctx->cipher->do_cipher(ctx,out,in,inl);
returns 0.


A bit of extra info, with cryptodev_verbosity=2, on 0.9 when the error 
occurs:

cryptodev: sshd[1205]: invalid session ID=0xAADBA6A0

With 1.7:
cryptodev: sshd[1520] (fill_kcop_from_cop:647): invalid session 
ID=0xEC91F39A

cryptodev: sshd[1520] (cryptodev_ioctl:857): Error copying from user



___

[Cryptodev-linux-devel] Problem with OpenSSH/OpenSSL Interaction When Cryptodev is Used

[Gordan Bobic]

I am experiencing a rather odd problem with cryptodev that I am not sure
where to follow next.

The distro is RedSleeve (EL6 armv5tel).
OpenSSH 5.3p1 (EL6 extra patches)
OpenSSL 1.0.1e (EL6 extra patches), rebuild with -DHAVE_CRYPTODEV 
(without digests)
Cryptodev 0.9 (yes, I know this is ancient but I tried 1.7 and it made 
no difference)
Linux 3.4.6 (old, but heavily patched for both the hardware 
compatibility and CVEs, binary that ships with my device)


What works:
mod_ssl with ECDHE-RSA-AES128-GCM-SHA256
openssl s_server <-> s_client

What doesn't:
sshd with offloadable ciphers
Non-offloadable ciphers (e.g. blowfish) always work fine.
aes128-cbc works without cryptodev module loaded, but when it is loaded 
the connection breaks _after_ sending the last login information.



With cryptodev loaded and Ciphers set to aes128-cbc:
On the client side, everything seems to work fine, right up to the end:

$ ssh localhost
[...]
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Last login: Tue May 26 16:40:17 2015 from localhost
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 client-session (t4 r0 i0/0 o0/0 fd 4/5 cc -1)

Connection to qnap closed by remote host.
Connection to qnap closed.
Transferred: sent 3472, received 4128 bytes, in 0.5 seconds
Bytes per second: sent 7622.1, received 9062.3
debug1: Exit status -1

This tells me that the connection setup actually succeeds, since this 
gets received by the client:

Last login: Tue May 26 16:40:17 2015 from localhost

On the server side with LogLevel DEBUG:
Everything looks OK initially:
[...]
May 26 16:59:49 qnap sshd[29520]: debug1: Allocating pty.
May 26 16:59:49 qnap sshd[29474]: debug1: session_new: session 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_pty_req: session 0 
alloc /dev/pts/4
May 26 16:59:49 qnap sshd[29520]: debug1: server_input_channel_req: 
channel 0 request env reply 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_by_channel: session 0 
channel 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_input_channel_req: 
session 0 req env
May 26 16:59:49 qnap sshd[29520]: debug1: server_input_channel_req: 
channel 0 request env reply 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_by_channel: session 0 
channel 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_input_channel_req: 
session 0 req env
May 26 16:59:49 qnap sshd[29520]: debug1: server_input_channel_req: 
channel 0 request shell reply 1
May 26 16:59:49 qnap sshd[29520]: debug1: session_by_channel: session 0 
channel 0
May 26 16:59:49 qnap sshd[29520]: debug1: session_input_channel_req: 
session 0 req shell
May 26 16:59:49 qnap sshd[29521]: debug1: Setting controlling tty using 
TIOCSCTTY.


Right up to here:

May 26 16:59:49 qnap sshd[29520]: fatal: evp_crypt: EVP_Cipher failed

The rest appears to be just clean-up:

May 26 16:59:49 qnap sshd[29520]: debug1: do_cleanup
May 26 16:59:49 qnap sshd[29474]: debug1: do_cleanup
May 26 16:59:49 qnap sshd[29474]: debug1: PAM: cleanup
May 26 16:59:49 qnap sshd[29474]: debug1: PAM: closing session
May 26 16:59:49 qnap sshd[29474]: pam_unix(sshd:session): session closed 
for user gordan

May 26 16:59:49 qnap sshd[29474]: debug1: PAM: deleting credentials
May 26 16:59:49 qnap sshd[29474]: debug1: session_pty_cleanup: session 0 
release /dev/pts/4



The error:
sshd[29520]: fatal: evp_crypt: EVP_Cipher failed
appears to be emitted by sshd rather than OpenSSL or
Cryptodev. The relevant bit of code is in OpenSSH's
cipher.c file:

void
cipher_crypt(CipherContext *cc, u_char *dest, const u_char *src, u_int
len)
{
 if (len % cc->cipher->block_size)
 fatal("cipher_encrypt: bad plaintext length %d", len);
 if (EVP_Cipher(&cc->evp, dest, (u_char *)src, len) == 0)
 fatal("evp_crypt: EVP_Cipher failed");
}

EVP_Cipher function is part of OpenSSL, declared in:
crypto/evp/evp.h:
int EVP_Cipher(EVP_CIPHER_CTX *c,
 unsigned char *out,
 const unsigned char *in,
 unsigned int inl);

and defined in:
crypto/evp/evp_lib.c:
int EVP_Cipher(EVP_CIPHER_CTX *ctx, unsigned char *out, const unsigned
char *in, unsigned int inl)
 {
#ifdef OPENSSL_FIPS
 FIPS_selftest_check();
#endif
 return ctx->cipher->do_cipher(ctx,out,in,inl);
 }

So it would appear that
ctx->cipher->do_cipher(ctx,out,in,inl);
returns 0.

Any advice on the best way to proceed with troubleshooting this would
be greatly appreciated.

Many thanks.

Gordan

___
Cryptodev-linux-devel mailing list
Cryptodev-linux-devel@gna.org
https://mail.gna.org/listinfo/cryptodev-linux-devel