Re: how to use the ktls
On Mon, Feb 03, 2020 at 10:49:38PM +, Rick Macklem wrote: > Benjamin Kaduk wrote: > >On Tue, Jan 28, 2020 at 11:01:31PM +, Rick Macklem wrote: > >> John Baldwin wrote: > >> [stuff snipped] > >> >I don't know yet. :-/ With the TOE-based TLS I had been testing with, > >> >this doesn't > >> >happen because the NIC blocks the data until it gets the key and then > >> >it's always > >> >available via KTLS. With software-based KTLS for RX (which I'm going to > >> >start > >> >working on soon), this won't be the case and you will potentially have > >> >some data > >> >already ready by OpenSSL that needs to be drained from OpenSSL before you > >> >can > >> >depend on KTLS. It's probably only the first few messsages, but I will > >> >need to figure > >> >out a way that you can tell how much pending data in userland you need to > >> >read via > >> >SSL_read() and then pass back into the kernel before relying on KTLS (it > >> >would just > >> >be a single chunk of data after SSL_connect you would have to do this > >> >for). > >> I think SSL_read() ends up calling ssl3_read_bytes(..APPLICATION..) and > >> then it throws > >> away non-application data records. (Not sure, ssl3_read_bytes() gets > >> pretty convoluted at > >> a glance.;-) > > > >Yes, SSL_read() interprets the TLS record type and only passes application > >data records through to the application. It doesn't exactly "throw away" > >the other records, though -- they still get processed, just internally to > >libssl :) > >I expect based on heuristics that the 485 bytes are a NewSessionTicket > >message, but that actual length is very much not a protocol constant and is > >an implementation detail of the TLS server. (That said, an openssl server > >is going to be producing the same length every time, for a given version of > >openssl, unless you configure it otherwise.) > Well, I looked at the data and it appears to be two application data records, > both of length 234. (These are in the receive queue before the other end does > an SSL_write() and the only data returned by SSL_read() is what a subsequent > SSL_write() has written.) > > My hunch is that, once they are unencrypted, they are just padding. > Anyhow, since they are "application data" the receive side of KERN_TLS > should be able to handle them. > --> I don't think I need to do anything after the SSL_connect() in userland > to deal with these. All wire-visible TLS 1.3 records after the initial handshake will appear with the content type of "application data"; there's an inner content type to indicate what is actually being conveyed: https://tools.ietf.org/html/rfc8446#section-5.2 . Two same-length records would match up with openssl sending two session tickets by default on a new connection. -Ben ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
Benjamin Kaduk wrote: >On Tue, Jan 28, 2020 at 11:01:31PM +, Rick Macklem wrote: >> John Baldwin wrote: >> [stuff snipped] >> >I don't know yet. :-/ With the TOE-based TLS I had been testing with, this >> >doesn't >> >happen because the NIC blocks the data until it gets the key and then it's >> >always >> >available via KTLS. With software-based KTLS for RX (which I'm going to >> >start >> >working on soon), this won't be the case and you will potentially have some >> >data >> >already ready by OpenSSL that needs to be drained from OpenSSL before you >> >can >> >depend on KTLS. It's probably only the first few messsages, but I will >> >need to figure >> >out a way that you can tell how much pending data in userland you need to >> >read via >> >SSL_read() and then pass back into the kernel before relying on KTLS (it >> >would just >> >be a single chunk of data after SSL_connect you would have to do this for). >> I think SSL_read() ends up calling ssl3_read_bytes(..APPLICATION..) and then >> it throws >> away non-application data records. (Not sure, ssl3_read_bytes() gets pretty >> convoluted at >> a glance.;-) > >Yes, SSL_read() interprets the TLS record type and only passes application >data records through to the application. It doesn't exactly "throw away" >the other records, though -- they still get processed, just internally to >libssl :) >I expect based on heuristics that the 485 bytes are a NewSessionTicket >message, but that actual length is very much not a protocol constant and is >an implementation detail of the TLS server. (That said, an openssl server >is going to be producing the same length every time, for a given version of >openssl, unless you configure it otherwise.) Well, I looked at the data and it appears to be two application data records, both of length 234. (These are in the receive queue before the other end does an SSL_write() and the only data returned by SSL_read() is what a subsequent SSL_write() has written.) My hunch is that, once they are unencrypted, they are just padding. Anyhow, since they are "application data" the receive side of KERN_TLS should be able to handle them. --> I don't think I need to do anything after the SSL_connect() in userland to deal with these. Thanks for your help, rick -Ben ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
On Tue, Jan 28, 2020 at 11:01:31PM +, Rick Macklem wrote: > John Baldwin wrote: > [stuff snipped] > >I don't know yet. :-/ With the TOE-based TLS I had been testing with, this > >doesn't > >happen because the NIC blocks the data until it gets the key and then it's > >always > >available via KTLS. With software-based KTLS for RX (which I'm going to > >start > >working on soon), this won't be the case and you will potentially have some > >data > >already ready by OpenSSL that needs to be drained from OpenSSL before you can > >depend on KTLS. It's probably only the first few messsages, but I will need > >to figure > >out a way that you can tell how much pending data in userland you need to > >read via > >SSL_read() and then pass back into the kernel before relying on KTLS (it > >would just > >be a single chunk of data after SSL_connect you would have to do this for). > I think SSL_read() ends up calling ssl3_read_bytes(..APPLICATION..) and then > it throws > away non-application data records. (Not sure, ssl3_read_bytes() gets pretty > convoluted at > a glance.;-) Yes, SSL_read() interprets the TLS record type and only passes application data records through to the application. It doesn't exactly "throw away" the other records, though -- they still get processed, just internally to libssl :) I expect based on heuristics that the 485 bytes are a NewSessionTicket message, but that actual length is very much not a protocol constant and is an implementation detail of the TLS server. (That said, an openssl server is going to be producing the same length every time, for a given version of openssl, unless you configure it otherwise.) -Ben ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
John Baldwin wrote: [stuff snipped] >I don't know yet. :-/ With the TOE-based TLS I had been testing with, this >doesn't >happen because the NIC blocks the data until it gets the key and then it's >always >available via KTLS. With software-based KTLS for RX (which I'm going to start >working on soon), this won't be the case and you will potentially have some >data >already ready by OpenSSL that needs to be drained from OpenSSL before you can >depend on KTLS. It's probably only the first few messsages, but I will need >to figure >out a way that you can tell how much pending data in userland you need to read >via >SSL_read() and then pass back into the kernel before relying on KTLS (it would >just >be a single chunk of data after SSL_connect you would have to do this for). I think SSL_read() ends up calling ssl3_read_bytes(..APPLICATION..) and then it throws away non-application data records. (Not sure, ssl3_read_bytes() gets pretty convoluted at a glance.;-) I've found another issue that should keep me amused for a while (this is becoming an interesting little project;-). The KERN_TLS needs unmapped pages on the mbuf chain, but that isn't what NFS generates. I think I'll have to implement some sort of copy function that creates mbufs with unmapped pages and then maps them into kernel space for long enough that the data can be copied, called just before sosend(). Most NFS RPC messages will easily fit in one page. Someday, the biggies like server read reply may be able to do what sendfile does and put the read data in unmapped page mbufs, avoiding the long list of mbuf clusters that VOP_READ() currently copies the data into. --> But that's longer term than getting this to work.;-) Thanks for all your help John, rick > I'm currently testing with a kernel that doesn't have options KERN_TLS and > (so long as I get rid of the 478 bytes), it then just does unencrypted RPCs. > > So, I guess the big question is can I get access to your WIP code for KTLS > receive? (I have no idea if I can make progress on it, but I can't do a lot > more > before I have that.) The WIP only works right now if you have a Chelsio T6 NIC as it uses the T6's TCP offload engine to do TLS. If you don't have that gear, ping me off-list. It would also let you not worry about the SSL_read case for now for initial testing. -- John Baldwin ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
On 1/9/20 2:53 PM, Rick Macklem wrote:
John Baldwin wrote:
On 1/7/20 3:02 PM, Rick Macklem wrote:
Someone once told me they were working on a netgraph node that did TLS
encapsulation of a stream.
I can not remember who it was, but I do remember they were dubious
about being allowed to give it back. :-(
I only mention this as it MAY be an architectural avenue to investigate.
Julian
Hi,
Now that I've completed NFSv4.2 I'm on to the next project, which is making NFS
work over TLS.
Of course, I know absolutely nothing about TLS, which will make this an
interesting
exercise for me.
I did find simple server code in the OpenSSL doc. which at least gives me a
starting
point for the initialization stuff.
As I understand it, this initialization must be done in userspace?
Then somehow, the ktls takes over and does the encryption of the
data being sent on the socket via sosend_generic(). Does that sound right?
So, how does the kernel know the stuff that the initialization phase (handshake)
figures out, or is it magic I don't have to worry about?
Don't waste much time replying to this. A few quick hints will keep me going for
now. (From what I've seen sofar, this TLS stuff isn't simple. And I thought
Kerberos
was a pain.;-)
Thanks in advance for any hints, rick
Hmmm, this might be a fair bit of work indeed.
If it was easy, it wouldn't be fun;-) FreeBSD13 is a ways off and if it
doesn't make that, oh well..
Right now KTLS only works for transmit (though I have some WIP for receive).
Hopefully your WIP will make progress someday, or I might be able to work on it.
KTLS does assumes that the initial handshake and key negotiation is handled by
OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel which
session keys to use.
Yea, I figured I'd need a daemon like the gssd for this. The krpc makes it a
little
more fun, since it handles TCP connections in the kernel.
I think what you would want to do is use something like OpenSSL_connect() in
userspace, and then check to see if KTLS "worked".
Thanks (and for the code below). I found the simple server code in the OpenSSL
doc,
but the client code gets a web page and is quite involved.
If it did, you can tell
the kernel it can write to the socket directly, otherwise you will have to
bounce data back out to userspace to run it through SSL_write() and have
userspace do SSL_read() and then feed data into the kernel.
I don't think bouncing the data up/down to/from userland would work well.
I'd say "if it can't be done in the kernel, too bad". The above could be used
for
a NULL RPC to see it is working, for the client.
The pseudo-code might look something like:
SSL *s;
s = SSL_new(...);
/* fd is the existing TCP socket */
SSL_set_fd(s, fd);
OpenSSL_connect(s);
if (BIO_get_ktls_send(SSL_get_wbio(s)) {
/* Can use KTLS for transmit. */
}
if (BIO_get_ktls_recv(SSL_get_rbio(s)) {
/* Can use KTLS for receive. */
}
Thanks John, rick
--
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
John Baldwin wrote:
>On 1/26/20 8:08 PM, Rick Macklem wrote:
>> John Baldwin wrote:
>> [stuff snipped]
>>> Hmmm, this might be a fair bit of work indeed.
>>>
>>> Right now KTLS only works for transmit (though I have some WIP for receive).
>>>
>>> KTLS does assumes that the initial handshake and key negotiation is handled
>>> by
>>> OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel which
>>> session keys to use.
>>>
>>> I think what you would want to do is use something like OpenSSL_connect() in
>>> userspace, and then check to see if KTLS "worked". If it did, you can tell
>>> the kernel it can write to the socket directly, otherwise you will have to
>>> bounce data back out to userspace to run it through SSL_write() and have
>>> userspace do SSL_read() and then feed data into the kernel.
>>>
>>> The pseudo-code might look something like:
>>>
>>> SSL *s;
>>>
>>> s = SSL_new(...);
>>>
>>> /* fd is the existing TCP socket */
>>> SSL_set_fd(s, fd);
>>> OpenSSL_connect(s);
>>> if (BIO_get_ktls_send(SSL_get_wbio(s)) {
>>>/* Can use KTLS for transmit. */
>>> }
>>> if (BIO_get_ktls_recv(SSL_get_rbio(s)) {
>>>/* Can use KTLS for receive. */
>>> }
>>
>> So, I've been making some progress. The first stab at the daemons that do the
>> handshake are now on svn in base/projects/nfs-over-tls/usr.sbin/rpctlscd and
>> rpctlssd.
>>
>> A couple of questions...
>> 1 - I haven't found BIO_get_ktls_send() or BIO_get_ktls_recv(). Are they in
>> some
>>different library?
>
>They only existing currently in OpenSSL master (which will be OpenSSL 3.0.0
>when it
>is released). I have some not-yet-tested WIP changes to backport those
>changes into
>the base OpenSSL, but it will also add overhead to future OpenSSL imports
>perhaps,
>so it is something I need to work with secteam@ on to decide if it's viable
>once I
>have a tested PoC.
>
>I will try to at least provide a patch to the security/openssl port to add a
>KTLS
>option "soon" that you could use for testing.
John, I wouldn't worry much about this.
The calls are currently #ifdef notnow in the daemon and I'm fine with that.
SSL_connect() has returned 1, so the daemon knows that the handshake is
complete and
the kernel code that did the upcall to the daemon can check for KERN_TLS
support.
>> 2 - After a successful SSL_connect(), the receive queue for the socket has
>> 478bytes
>> of stuff in it. SSL_read() seems to know how to skip over it, but I
>> haven't
>> figured out a good way to do this. (I currently just do a
>> recv(..478,0) on the
>> socket.)
>> Any idea what to do with this? (Or will the receive side of the ktls
>> figure out
>> how to skip over it?)
>
>I don't know yet. :-/ With the TOE-based TLS I had been testing with, this
>doesn't
>happen because the NIC blocks the data until it gets the key and then it's
>always
>available via KTLS. With software-based KTLS for RX (which I'm going to start
>working on soon), this won't be the case and you will potentially have some
>data
>already ready by OpenSSL that needs to be drained from OpenSSL before you can
>depend on KTLS. It's probably only the first few messsages, but I will need
>to figure
>out a way that you can tell how much pending data in userland you need to read
>via
>SSL_read() and then pass back into the kernel before relying on KTLS (it would
>just
>be a single chunk of data after SSL_connect you would have to do this for).
Well, SSL_read() doesn't return these bytes. I think it just throws them away.
I have a simple test client/server where the client sends "HELLO THERE" to the
server and the server replies "GOODBYE" after the SSL_connect()/SSL_accept()
has been done.
--> If the "HELLO THERE"/"GOODBYE" is done with SSL_write()/SSL_read() it works.
however
--> If the above is done with send()/recv(), the server gets the "HELLO THERE",
but
the client gets 485bytes of data, where the last 7 are "GOODBYE".
--> If I do a recv( ..475..) in the client right after SSL_connect() it
works ok.
I do this for testing, since it can then do the NFS mount (unencrypted).
Looking inside SSL_read() I found:
*
1742 * If we are a client and haven't received the ServerHello etc then
we
1743 * better do that
1744 */
1745ossl_statem_check_finish_init(s, 0);
but all ossl_statem_check_finish_init(s, 0); seems to do is set a variable
"in_init = 1".
Then it calls the method->read() function, which I'd guess is in the crypto code
and it seems to get rid of these bytes. (I hate trying to figure out what calls
what
for object oriented code;-).
Btw. SSL_is_finished_init() returns 1 right after the SSL_connect(), so it
seems to
think the hadnshake is done, although it hasn't read these 478 bytes from the
server.
Anyhow, I'd guess the TOE engine knows how to do get rid of this stuff like
SSL_read()
does?
>> I'm currently testing with a kernel that doesn't have options KERN_TLS and
>> (so long as I ge
Re: how to use the ktls
On Mon, Jan 27, 2020 at 8:40 AM Freddie Cash wrote: > On Sun, Jan 26, 2020 at 12:08 PM Rick Macklem > wrote: > >> Oh, and for anyone out there... >> What is the easiest freebie way to test signed certificates? >> (I currently am using a self-signed certificate, but I need to test the >> "real" version >> at some point soon.) >> > > Let's Encrypt is what you are looking for. Create real, signed, > certificates, for free. They're only good for 90 days, but they are easy > to renew. There's various script and programs out there for managing Let's > Encrypt certificates (certbot, acme.sh, dehydrated, etc). There's a bunch > of different bits available in the ports tree. > > We use dehydrated at work, using DNS for authenticating the cert requests, > and have it full automated via cron, managing certs for 50-odd domains > (school servers and firewalls). Works great. > Forgot the link: https://letsencrypt.org -- Freddie Cash fjwc...@gmail.com ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
On Sun, Jan 26, 2020 at 12:08 PM Rick Macklem wrote: > Oh, and for anyone out there... > What is the easiest freebie way to test signed certificates? > (I currently am using a self-signed certificate, but I need to test the > "real" version > at some point soon.) > Let's Encrypt is what you are looking for. Create real, signed, certificates, for free. They're only good for 90 days, but they are easy to renew. There's various script and programs out there for managing Let's Encrypt certificates (certbot, acme.sh, dehydrated, etc). There's a bunch of different bits available in the ports tree. We use dehydrated at work, using DNS for authenticating the cert requests, and have it full automated via cron, managing certs for 50-odd domains (school servers and firewalls). Works great. -- Freddie Cash fjwc...@gmail.com ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
On 1/26/20 8:08 PM, Rick Macklem wrote:
John Baldwin wrote:
[stuff snipped]
Hmmm, this might be a fair bit of work indeed.
Right now KTLS only works for transmit (though I have some WIP for receive).
KTLS does assumes that the initial handshake and key negotiation is handled by
OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel which
session keys to use.
I think what you would want to do is use something like OpenSSL_connect() in
userspace, and then check to see if KTLS "worked". If it did, you can tell
the kernel it can write to the socket directly, otherwise you will have to
bounce data back out to userspace to run it through SSL_write() and have
userspace do SSL_read() and then feed data into the kernel.
The pseudo-code might look something like:
SSL *s;
s = SSL_new(...);
/* fd is the existing TCP socket */
SSL_set_fd(s, fd);
OpenSSL_connect(s);
if (BIO_get_ktls_send(SSL_get_wbio(s)) {
/* Can use KTLS for transmit. */
}
if (BIO_get_ktls_recv(SSL_get_rbio(s)) {
/* Can use KTLS for receive. */
}
So, I've been making some progress. The first stab at the daemons that do the
handshake are now on svn in base/projects/nfs-over-tls/usr.sbin/rpctlscd and
rpctlssd.
A couple of questions...
1 - I haven't found BIO_get_ktls_send() or BIO_get_ktls_recv(). Are they in some
different library?
They only existing currently in OpenSSL master (which will be OpenSSL 3.0.0
when it
is released). I have some not-yet-tested WIP changes to backport those changes
into
the base OpenSSL, but it will also add overhead to future OpenSSL imports
perhaps,
so it is something I need to work with secteam@ on to decide if it's viable
once I
have a tested PoC.
I will try to at least provide a patch to the security/openssl port to add a
KTLS
option "soon" that you could use for testing.
2 - After a successful SSL_connect(), the receive queue for the socket has
478bytes
of stuff in it. SSL_read() seems to know how to skip over it, but I
haven't
figured out a good way to do this. (I currently just do a recv(..478,0)
on the
socket.)
Any idea what to do with this? (Or will the receive side of the ktls
figure out
how to skip over it?)
I don't know yet. :-/ With the TOE-based TLS I had been testing with, this
doesn't
happen because the NIC blocks the data until it gets the key and then it's
always
available via KTLS. With software-based KTLS for RX (which I'm going to start
working on soon), this won't be the case and you will potentially have some data
already ready by OpenSSL that needs to be drained from OpenSSL before you can
depend on KTLS. It's probably only the first few messsages, but I will need to
figure
out a way that you can tell how much pending data in userland you need to read
via
SSL_read() and then pass back into the kernel before relying on KTLS (it would
just
be a single chunk of data after SSL_connect you would have to do this for).
I'm currently testing with a kernel that doesn't have options KERN_TLS and
(so long as I get rid of the 478 bytes), it then just does unencrypted RPCs.
So, I guess the big question is can I get access to your WIP code for KTLS
receive? (I have no idea if I can make progress on it, but I can't do a lot more
before I have that.)
The WIP only works right now if you have a Chelsio T6 NIC as it uses the T6's
TCP
offload engine to do TLS. If you don't have that gear, ping me off-list. It
would also let you not worry about the SSL_read case for now for initial
testing.
--
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
John Baldwin wrote:
[stuff snipped]
>Hmmm, this might be a fair bit of work indeed.
>
>Right now KTLS only works for transmit (though I have some WIP for receive).
>
>KTLS does assumes that the initial handshake and key negotiation is handled by
>OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel which
>session keys to use.
>
>I think what you would want to do is use something like OpenSSL_connect() in
>userspace, and then check to see if KTLS "worked". If it did, you can tell
>the kernel it can write to the socket directly, otherwise you will have to
>bounce data back out to userspace to run it through SSL_write() and have
>userspace do SSL_read() and then feed data into the kernel.
>
>The pseudo-code might look something like:
>
>SSL *s;
>
>s = SSL_new(...);
>
>/* fd is the existing TCP socket */
>SSL_set_fd(s, fd);
>OpenSSL_connect(s);
>if (BIO_get_ktls_send(SSL_get_wbio(s)) {
> /* Can use KTLS for transmit. */
>}
>if (BIO_get_ktls_recv(SSL_get_rbio(s)) {
> /* Can use KTLS for receive. */
>}
So, I've been making some progress. The first stab at the daemons that do the
handshake are now on svn in base/projects/nfs-over-tls/usr.sbin/rpctlscd and
rpctlssd.
A couple of questions...
1 - I haven't found BIO_get_ktls_send() or BIO_get_ktls_recv(). Are they in some
different library?
2 - After a successful SSL_connect(), the receive queue for the socket has
478bytes
of stuff in it. SSL_read() seems to know how to skip over it, but I haven't
figured out a good way to do this. (I currently just do a recv(..478,0) on
the
socket.)
Any idea what to do with this? (Or will the receive side of the ktls
figure out
how to skip over it?)
I'm currently testing with a kernel that doesn't have options KERN_TLS and
(so long as I get rid of the 478 bytes), it then just does unencrypted RPCs.
So, I guess the big question is can I get access to your WIP code for KTLS
receive? (I have no idea if I can make progress on it, but I can't do a lot more
before I have that.)
Oh, and for anyone out there...
What is the easiest freebie way to test signed certificates?
(I currently am using a self-signed certificate, but I need to test the "real"
version
at some point soon.)
Thanks, rick
--
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
John Baldwin wrote: >On 1/12/20 8:23 PM, Benjamin Kaduk wrote: >> On Thu, Jan 09, 2020 at 10:53:38PM +, Rick Macklem wrote: >>> John Baldwin wrote: On 1/7/20 3:02 PM, Rick Macklem wrote: > Hi, > > Now that I've completed NFSv4.2 I'm on to the next project, which is > making NFS > work over TLS. > Of course, I know absolutely nothing about TLS, which will make this an > interesting > exercise for me. > I did find simple server code in the OpenSSL doc. which at least gives me > a starting > point for the initialization stuff. > As I understand it, this initialization must be done in userspace? > > Then somehow, the ktls takes over and does the encryption of the > data being sent on the socket via sosend_generic(). Does that sound right? > > So, how does the kernel know the stuff that the initialization phase > (handshake) > figures out, or is it magic I don't have to worry about? > > Don't waste much time replying to this. A few quick hints will keep me > going for > now. (From what I've seen sofar, this TLS stuff isn't simple. And I > thought Kerberos > was a pain.;-) > > Thanks in advance for any hints, rick Hmmm, this might be a fair bit of work indeed. >>> If it was easy, it wouldn't be fun;-) FreeBSD13 is a ways off and if it >>> doesn't make that, oh well.. >>> Right now KTLS only works for transmit (though I have some WIP for receive). >>> Hopefully your WIP will make progress someday, or I might be able to work >>> on it. >>> KTLS does assumes that the initial handshake and key negotiation is handled by OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel which session keys to use. >>> Yea, I figured I'd need a daemon like the gssd for this. The krpc makes it >>> a little >>> more fun, since it handles TCP connections in the kernel. >>> I think what you would want to do is use something like OpenSSL_connect() in userspace, and then check to see if KTLS "worked". >>> Thanks (and for the code below). I found the simple server code in the >>> OpenSSL doc, >>> but the client code gets a web page and is quite involved. >>> If it did, you can tell the kernel it can write to the socket directly, otherwise you will have to bounce data back out to userspace to run it through SSL_write() and have userspace do SSL_read() and then feed data into the kernel. >>> I don't think bouncing the data up/down to/from userland would work well. >>> I'd say "if it can't be done in the kernel, too bad". The above could be >>> used for >>> a NULL RPC to see it is working, for the client. >> >> So you're saying that we'd only support rpc-over-tls as an NFS client and >> not as a server, at least until the WIP for ktls read appears? Actually, I'd say that neither NFS client nor server will work over tls until the receive side works, since NFS RPCs result in bi-directional traffic. >To be clear, I have KTLS RX working with TOE right now. I have a design in my >head for KTLS RX that would use software and co-processor engines via OCF such >as aesni(4) and ccr(4) that I hope to implement in the next few months, so KTLS >RX isn't too far off. OpenSSL already supports KTLS RX on Linux and the >FreeBSD >patches I already have use the same API. (Each received TLS frame is read via >recvmsg() with the TLS header fields in a cmsg.) Sounds good. It will be a while before I get to the stage where I need it. I'm currently working on how to give userland access to a socket created in the kernel, so that a daemon can use it. Have fun with it, rick -- John Baldwin ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
On 1/12/20 8:23 PM, Benjamin Kaduk wrote: > On Thu, Jan 09, 2020 at 10:53:38PM +, Rick Macklem wrote: >> John Baldwin wrote: >>> On 1/7/20 3:02 PM, Rick Macklem wrote: Hi, Now that I've completed NFSv4.2 I'm on to the next project, which is making NFS work over TLS. Of course, I know absolutely nothing about TLS, which will make this an interesting exercise for me. I did find simple server code in the OpenSSL doc. which at least gives me a starting point for the initialization stuff. As I understand it, this initialization must be done in userspace? Then somehow, the ktls takes over and does the encryption of the data being sent on the socket via sosend_generic(). Does that sound right? So, how does the kernel know the stuff that the initialization phase (handshake) figures out, or is it magic I don't have to worry about? Don't waste much time replying to this. A few quick hints will keep me going for now. (From what I've seen sofar, this TLS stuff isn't simple. And I thought Kerberos was a pain.;-) Thanks in advance for any hints, rick >>> >>> Hmmm, this might be a fair bit of work indeed. >> If it was easy, it wouldn't be fun;-) FreeBSD13 is a ways off and if it >> doesn't make that, oh well.. >> >>> Right now KTLS only works for transmit (though I have some WIP for receive). >> Hopefully your WIP will make progress someday, or I might be able to work on >> it. >> >>> KTLS does assumes that the initial handshake and key negotiation is handled >>> by >>> OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel which >>> session keys to use. >> Yea, I figured I'd need a daemon like the gssd for this. The krpc makes it a >> little >> more fun, since it handles TCP connections in the kernel. >> >>> I think what you would want to do is use something like OpenSSL_connect() in >>> userspace, and then check to see if KTLS "worked". >> Thanks (and for the code below). I found the simple server code in the >> OpenSSL doc, >> but the client code gets a web page and is quite involved. >> >>> If it did, you can tell >>> the kernel it can write to the socket directly, otherwise you will have to >>> bounce data back out to userspace to run it through SSL_write() and have >>> userspace do SSL_read() and then feed data into the kernel. >> I don't think bouncing the data up/down to/from userland would work well. >> I'd say "if it can't be done in the kernel, too bad". The above could be >> used for >> a NULL RPC to see it is working, for the client. > > So you're saying that we'd only support rpc-over-tls as an NFS client and > not as a server, at least until the WIP for ktls read appears? To be clear, I have KTLS RX working with TOE right now. I have a design in my head for KTLS RX that would use software and co-processor engines via OCF such as aesni(4) and ccr(4) that I hope to implement in the next few months, so KTLS RX isn't too far off. OpenSSL already supports KTLS RX on Linux and the FreeBSD patches I already have use the same API. (Each received TLS frame is read via recvmsg() with the TLS header fields in a cmsg.) -- John Baldwin ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
On Thu, Jan 09, 2020 at 10:53:38PM +, Rick Macklem wrote: > John Baldwin wrote: > >On 1/7/20 3:02 PM, Rick Macklem wrote: > >> Hi, > >> > >> Now that I've completed NFSv4.2 I'm on to the next project, which is > >> making NFS > >> work over TLS. > >> Of course, I know absolutely nothing about TLS, which will make this an > >> interesting > >> exercise for me. > >> I did find simple server code in the OpenSSL doc. which at least gives me > >> a starting > >> point for the initialization stuff. > >> As I understand it, this initialization must be done in userspace? > >> > >> Then somehow, the ktls takes over and does the encryption of the > >> data being sent on the socket via sosend_generic(). Does that sound right? > >> > >> So, how does the kernel know the stuff that the initialization phase > >> (handshake) > >> figures out, or is it magic I don't have to worry about? > >> > >> Don't waste much time replying to this. A few quick hints will keep me > >> going for > >> now. (From what I've seen sofar, this TLS stuff isn't simple. And I > >> thought Kerberos > >> was a pain.;-) > >> > >> Thanks in advance for any hints, rick > > > >Hmmm, this might be a fair bit of work indeed. > If it was easy, it wouldn't be fun;-) FreeBSD13 is a ways off and if it > doesn't make that, oh well.. > > >Right now KTLS only works for transmit (though I have some WIP for receive). > Hopefully your WIP will make progress someday, or I might be able to work on > it. > > >KTLS does assumes that the initial handshake and key negotiation is handled > >by > >OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel which > >session keys to use. > Yea, I figured I'd need a daemon like the gssd for this. The krpc makes it a > little > more fun, since it handles TCP connections in the kernel. > > >I think what you would want to do is use something like OpenSSL_connect() in > >userspace, and then check to see if KTLS "worked". > Thanks (and for the code below). I found the simple server code in the > OpenSSL doc, > but the client code gets a web page and is quite involved. > > >If it did, you can tell > >the kernel it can write to the socket directly, otherwise you will have to > >bounce data back out to userspace to run it through SSL_write() and have > >userspace do SSL_read() and then feed data into the kernel. > I don't think bouncing the data up/down to/from userland would work well. > I'd say "if it can't be done in the kernel, too bad". The above could be used > for > a NULL RPC to see it is working, for the client. So you're saying that we'd only support rpc-over-tls as an NFS client and not as a server, at least until the WIP for ktls read appears? -Ben ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
John Baldwin wrote:
>On 1/7/20 3:02 PM, Rick Macklem wrote:
>> Hi,
>>
>> Now that I've completed NFSv4.2 I'm on to the next project, which is making
>> NFS
>> work over TLS.
>> Of course, I know absolutely nothing about TLS, which will make this an
>> interesting
>> exercise for me.
>> I did find simple server code in the OpenSSL doc. which at least gives me a
>> starting
>> point for the initialization stuff.
>> As I understand it, this initialization must be done in userspace?
>>
>> Then somehow, the ktls takes over and does the encryption of the
>> data being sent on the socket via sosend_generic(). Does that sound right?
>>
>> So, how does the kernel know the stuff that the initialization phase
>> (handshake)
>> figures out, or is it magic I don't have to worry about?
>>
>> Don't waste much time replying to this. A few quick hints will keep me going
>> for
>> now. (From what I've seen sofar, this TLS stuff isn't simple. And I thought
>> Kerberos
>> was a pain.;-)
>>
>> Thanks in advance for any hints, rick
>
>Hmmm, this might be a fair bit of work indeed.
If it was easy, it wouldn't be fun;-) FreeBSD13 is a ways off and if it
doesn't make that, oh well..
>Right now KTLS only works for transmit (though I have some WIP for receive).
Hopefully your WIP will make progress someday, or I might be able to work on it.
>KTLS does assumes that the initial handshake and key negotiation is handled by
>OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel which
>session keys to use.
Yea, I figured I'd need a daemon like the gssd for this. The krpc makes it a
little
more fun, since it handles TCP connections in the kernel.
>I think what you would want to do is use something like OpenSSL_connect() in
>userspace, and then check to see if KTLS "worked".
Thanks (and for the code below). I found the simple server code in the OpenSSL
doc,
but the client code gets a web page and is quite involved.
>If it did, you can tell
>the kernel it can write to the socket directly, otherwise you will have to
>bounce data back out to userspace to run it through SSL_write() and have
>userspace do SSL_read() and then feed data into the kernel.
I don't think bouncing the data up/down to/from userland would work well.
I'd say "if it can't be done in the kernel, too bad". The above could be used
for
a NULL RPC to see it is working, for the client.
>The pseudo-code might look something like:
>
>SSL *s;
>
>s = SSL_new(...);
>
>/* fd is the existing TCP socket */
>SSL_set_fd(s, fd);
>OpenSSL_connect(s);
>if (BIO_get_ktls_send(SSL_get_wbio(s)) {
> /* Can use KTLS for transmit. */
>}
>if (BIO_get_ktls_recv(SSL_get_rbio(s)) {
> /* Can use KTLS for receive. */
>}
Thanks John, rick
--
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: how to use the ktls
On 1/7/20 3:02 PM, Rick Macklem wrote:
> Hi,
>
> Now that I've completed NFSv4.2 I'm on to the next project, which is making
> NFS
> work over TLS.
> Of course, I know absolutely nothing about TLS, which will make this an
> interesting
> exercise for me.
> I did find simple server code in the OpenSSL doc. which at least gives me a
> starting
> point for the initialization stuff.
> As I understand it, this initialization must be done in userspace?
>
> Then somehow, the ktls takes over and does the encryption of the
> data being sent on the socket via sosend_generic(). Does that sound right?
>
> So, how does the kernel know the stuff that the initialization phase
> (handshake)
> figures out, or is it magic I don't have to worry about?
>
> Don't waste much time replying to this. A few quick hints will keep me going
> for
> now. (From what I've seen sofar, this TLS stuff isn't simple. And I thought
> Kerberos
> was a pain.;-)
>
> Thanks in advance for any hints, rick
Hmmm, this might be a fair bit of work indeed.
Right now KTLS only works for transmit (though I have some WIP for receive).
KTLS does assumes that the initial handshake and key negotiation is handled by
OpenSSL. OpenSSL uses custom setockopt() calls to tell the kernel which
session keys to use.
I think what you would want to do is use something like OpenSSL_connect() in
userspace, and then check to see if KTLS "worked". If it did, you can tell
the kernel it can write to the socket directly, otherwise you will have to
bounce data back out to userspace to run it through SSL_write() and have
userspace do SSL_read() and then feed data into the kernel.
The pseudo-code might look something like:
SSL *s;
s = SSL_new(...);
/* fd is the existing TCP socket */
SSL_set_fd(s, fd);
OpenSSL_connect(s);
if (BIO_get_ktls_send(SSL_get_wbio(s)) {
/* Can use KTLS for transmit. */
}
if (BIO_get_ktls_recv(SSL_get_rbio(s)) {
/* Can use KTLS for receive. */
}
--
John Baldwin
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
how to use the ktls
Hi, Now that I've completed NFSv4.2 I'm on to the next project, which is making NFS work over TLS. Of course, I know absolutely nothing about TLS, which will make this an interesting exercise for me. I did find simple server code in the OpenSSL doc. which at least gives me a starting point for the initialization stuff. As I understand it, this initialization must be done in userspace? Then somehow, the ktls takes over and does the encryption of the data being sent on the socket via sosend_generic(). Does that sound right? So, how does the kernel know the stuff that the initialization phase (handshake) figures out, or is it magic I don't have to worry about? Don't waste much time replying to this. A few quick hints will keep me going for now. (From what I've seen sofar, this TLS stuff isn't simple. And I thought Kerberos was a pain.;-) Thanks in advance for any hints, rick ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
