RE: Can't link a static library with custom OpenSSL rsa engine

[Scott Neugroschl]

You need to put the static library at the END of your link command.  A static 
library is searched when it is encountered in the link stream, and only the 
items needed will be used from it.

Because you have it first, there are no undefined symbols, and no items will be 
used from it.

From: openssl-users  On Behalf Of Shariful 
Alam
Sent: Tuesday, November 17, 2020 12:40 PM
To: openssl-users@openssl.org
Subject: Can't link a static library with custom OpenSSL rsa engine

Hello,
I have a custom rsa engine. It builds and works fine. Later, I have added a 
static library with my custom engine code. My code compiles. However, when I 
try to load the custom engine it shows invalid engine "rsa-engine-new".  The 
full error is given below,
x@x:~/Downloads/x/x/x/rsa_engine$ openssl rsautl -decrypt -inkey private.pem 
-in msg.enc -engine rsa-engine-new
invalid engine "rsa-engine-new"
140112744122112:error:25066067:DSO support routines:dlfcn_load:could not load 
the shared 
library:crypto/dso/dso_dlfcn.c:119:filename(/opt/openssl/lib/engines-1.1/rsa-engine-new.so):
 
/opt/openssl/lib/engines-1.1/rsa-engine-new.so:
 undefined symbol: dune_init
140112744122112:error:25070067:DSO support routines:DSO_load:could not load the 
shared library:crypto/dso/dso_lib.c:162:
140112744122112:error:260B6084:engine routines:dynamic_load:dso not 
found:crypto/engine/eng_dyn.c:414:
140112744122112:error:2606A074:engine routines:ENGINE_by_id:no such 
engine:crypto/engine/eng_list.c:334:id=rsa-engine-new
140112744122112:error:25066067:DSO support routines:dlfcn_load:could not load 
the shared 
library:crypto/dso/dso_dlfcn.c:119:filename(librsa-engine-new.so):
 
librsa-engine-new.so:
 cannot open shared object file: No such file or directory
140112744122112:error:25070067:DSO support routines:DSO_load:could not load the 
shared library:crypto/dso/dso_lib.c:162:
140112744122112:error:260B6084:engine routines:dynamic_load:dso not 
found:crypto/engine/eng_dyn.c:414:

Now the error doesn't say much about the cause of invalid engine. However my 
guess is it is from the  "undefined symbol: dune_init". "dune_init" is from the 
static library. Therefire I believe my linking is not working. I use the 
following Makefile to compile the engine,

  1.  rsa-engine: rsa/rsa.c rsa/bignum.c rsa/aes.c rsa/x509parse.c rsa/pem.c
  2.  gcc -fPIC -o rsa/rsa.o -c rsa/rsa.c
  3.  gcc -fPIC -o rsa/bignum.o -c rsa/bignum.c
  4.  gcc -fPIC -o rsa/aes.o -c rsa/aes.c
  5.  gcc -fPIC -o rsa/x509parse.o -c rsa/x509parse.c
  6.  gcc -fPIC -o rsa/pem.o -c rsa/pem.c
  7.  gcc -fPIC -c rsa-engine.c
  8.  gcc -shared -o 
librsa_engine.so
 libdune/libdune.a -lcrypto rsa-engine.o rsa/rsa.o rsa/bignum.o rsa/aes.o 
rsa/x509parse.o rsa/pem.o
  9.  mv 
librsa_engine.so
 
rsa-engine-new.so
  10. sudo cp 
rsa-engine-new.so
 /opt/openssl/lib/engines-1.1/
  11. clean:
  12. rm -f *.o rsa/*.o *.so rsa-engine
So, can anyone please if my guess is correct or not? If my guess is correct, 
how can I fix my Makefile?

N.B: Static library

  *   libdune/libdune.a is in the same directory with the main rsa-engine.c
  *   libdune/libdune.a is compiled with -fPIC flag

Thanks,
Shariful

RE: Can't link a static library with custom OpenSSL rsa engine

[Heberlein, Kurt William]

You might try changing this:


8. gcc -shared -o librsa_engine.so libdune/libdune.a -lcrypto rsa-engine.o 
rsa/rsa.o rsa/bignum.o rsa/aes.o rsa/x509parse.o rsa/pem.o
to this:
   gcc –shared –o librsa_engine.so –L./libdune rsa_engine.o 
rsa/rsa.o rsa/bignum.o rsa/aes.o rsa/x509parse.o rsa/pem.o –Wl,-Bstatic 
–llibdune –Wl,-Bdynamic –lcrypto

just a guess.  cheers

Can't link a static library with custom OpenSSL rsa engine

[Shariful Alam]

Hello,
I have a custom rsa engine. It builds and works fine. Later, I have added a
static library with my custom engine code. My code compiles. However, when
I try to load the custom engine it shows *invalid engine "rsa-engine-new".
*The full error is given below,

x@x:~/Downloads/x/x/x/rsa_engine$ openssl rsautl -decrypt -inkey
private.pem -in msg.enc -engine rsa-engine-new
invalid engine "rsa-engine-new"
140112744122112:error:25066067:DSO support routines:dlfcn_load:could not
load the shared
library:crypto/dso/dso_dlfcn.c:119:filename(/opt/openssl/lib/engines-1.1/rsa-engine-new.so):
/opt/openssl/lib/engines-1.1/rsa-engine-new.so: undefined symbol: dune_init
140112744122112:error:25070067:DSO support routines:DSO_load:could not load
the shared library:crypto/dso/dso_lib.c:162:
140112744122112:error:260B6084:engine routines:dynamic_load:dso not
found:crypto/engine/eng_dyn.c:414:
140112744122112:error:2606A074:engine routines:ENGINE_by_id:no such
engine:crypto/engine/eng_list.c:334:id=rsa-engine-new
140112744122112:error:25066067:DSO support routines:dlfcn_load:could not
load the shared
library:crypto/dso/dso_dlfcn.c:119:filename(librsa-engine-new.so):
librsa-engine-new.so: cannot open shared object file: No such file or
directory
140112744122112:error:25070067:DSO support routines:DSO_load:could not load
the shared library:crypto/dso/dso_lib.c:162:
140112744122112:error:260B6084:engine routines:dynamic_load:dso not
found:crypto/engine/eng_dyn.c:414:


Now the error doesn't say much about the cause of invalid engine. However
my guess is it is from the  "*undefined symbol: dune_init*". "dune_init" is
from the static library. Therefire I believe my linking is not working. I
use the following Makefile to compile the engine,


   1. rsa-engine: rsa/rsa.c rsa/bignum.c rsa/aes.c rsa/x509parse.c rsa/pem.c
   2. gcc -fPIC -o rsa/rsa.o -c rsa/rsa.c
   3. gcc -fPIC -o rsa/bignum.o -c rsa/bignum.c
   4. gcc -fPIC -o rsa/aes.o -c rsa/aes.c
   5. gcc -fPIC -o rsa/x509parse.o -c rsa/x509parse.c
   6. gcc -fPIC -o rsa/pem.o -c rsa/pem.c
   7. gcc -fPIC -c rsa-engine.c
   8. gcc -shared -o librsa_engine.so libdune/libdune.a -lcrypto
   rsa-engine.o rsa/rsa.o rsa/bignum.o rsa/aes.o rsa/x509parse.o rsa/pem.o
   9. mv librsa_engine.so rsa-engine-new.so
   10. sudo cp rsa-engine-new.so /opt/openssl/lib/engines-1.1/
   11. clean:
   12. rm -f *.o rsa/*.o *.so rsa-engine

So, can anyone please if my guess is correct or not? If my guess is
correct, how can I fix my Makefile?

N.B: Static library

   - libdune/libdune.a is in the same directory with the main rsa-engine.c
   - libdune/libdune.a is compiled with -fPIC flag


Thanks,
Shariful

Re: Server application hangs on SS_read, even when client disconnects

[Matt Caswell]

On 17/11/2020 13:56, Michael Wojcik wrote:
>> From: Kyle Hamilton 
>> Sent: Tuesday, 17 November, 2020 02:37
>> On Fri, Nov 13, 2020 at 11:51 AM Michael Wojcik
>>  wrote:
>>>
 From: Brice André 
 Sent: Friday, 13 November, 2020 09:13
>>>
 "Does the server parent process close its copy of the conversation socket?"
 I checked in my code, but it seems that no. Is it needed?
>>>
>>> You'll want to do it, for a few reasons: ...
>>
>> There's another reason why you'll want to close your socket with
>> SSL_close(): SSL (and TLS) view a prematurely-closed stream as an
>> exceptional condition to be reported to the application. This is to
>> prevent truncation attacks against the data communication layer.
>> While your application may not need that level of protection, it helps
>> to keep the state of your application in lockstep with the state of
>> the TLS protocol.  If your application doesn't expect to send any more
>> data, SSL_close() sends another record across the TCP connection to
>> tell the remote side that it should not keep the descriptor open.
> 
> This is true, but not what we're talking about here. When the
> application is done with the conversation, it should use SSL_close
> to terminate the conversation.
> 
> Here, though, we're talking about the server parent process closing
> its descriptor for the socket after forking the child process. At that
> point the application is not done with the conversation, and calling
> SSL_close in the server would be a mistake.
> 
> Now, if the server is unable to start a child process (e.g. fork fails
> because the user's process limit has been reached), or if for whatever
> other reason it decides to terminate the conversation without further
> processing, SSL_close would be appropriate.

Just for clarity, there is no such function as SSL_close. I assume
SSL_shutdown is what people mean.

Matt

RE: Server application hangs on SS_read, even when client disconnects

[Michael Wojcik]

> From: Kyle Hamilton 
> Sent: Tuesday, 17 November, 2020 02:37
> On Fri, Nov 13, 2020 at 11:51 AM Michael Wojcik
>  wrote:
> >
> > > From: Brice André 
> > > Sent: Friday, 13 November, 2020 09:13
> >
> > > "Does the server parent process close its copy of the conversation 
> > > socket?"
> > > I checked in my code, but it seems that no. Is it needed?
> >
> > You'll want to do it, for a few reasons: ...
>
> There's another reason why you'll want to close your socket with
> SSL_close(): SSL (and TLS) view a prematurely-closed stream as an
> exceptional condition to be reported to the application. This is to
> prevent truncation attacks against the data communication layer.
> While your application may not need that level of protection, it helps
> to keep the state of your application in lockstep with the state of
> the TLS protocol.  If your application doesn't expect to send any more
> data, SSL_close() sends another record across the TCP connection to
> tell the remote side that it should not keep the descriptor open.

This is true, but not what we're talking about here. When the
application is done with the conversation, it should use SSL_close
to terminate the conversation.

Here, though, we're talking about the server parent process closing
its descriptor for the socket after forking the child process. At that
point the application is not done with the conversation, and calling
SSL_close in the server would be a mistake.

Now, if the server is unable to start a child process (e.g. fork fails
because the user's process limit has been reached), or if for whatever
other reason it decides to terminate the conversation without further
processing, SSL_close would be appropriate.

--
Michael Wojcik

Re: Server application hangs on SS_read, even when client disconnects

[Kyle Hamilton]

There's another reason why you'll want to close your socket with
SSL_close(): SSL (and TLS) view a prematurely-closed stream as an
exceptional condition to be reported to the application. This is to
prevent truncation attacks against the data communication layer.
While your application may not need that level of protection, it helps
to keep the state of your application in lockstep with the state of
the TLS protocol.  If your application doesn't expect to send any more
data, SSL_close() sends another record across the TCP connection to
tell the remote side that it should not keep the descriptor open.

-Kyle H

On Fri, Nov 13, 2020 at 11:51 AM Michael Wojcik
 wrote:
>
> > From: Brice André 
> > Sent: Friday, 13 November, 2020 09:13
>
> > "Does the server parent process close its copy of the conversation socket?"
> > I checked in my code, but it seems that no. Is it needed?
>
> You'll want to do it, for a few reasons:
>
> - You'll be leaking descriptors in the server, and eventually it will hit its 
> limit.
> - If the child process dies without cleanly closing its end of the 
> conversation,
> the parent will still have an open descriptor for the socket, so the network 
> stack
> won't terminate the TCP connection.
> - A related problem: If the child just closes its socket without calling 
> shutdown,
> no FIN will be sent to the client system (because the parent still has its 
> copy of
> the socket open). The client system will have the connection in one of the 
> termination
> states (FIN_WAIT, maybe? I don't have my references handy) until it times out.
> - A bug in the parent process might cause it to operate on the connected 
> socket,
> causing unexpected traffic on the connection.
> - All such sockets will be inherited by future child processes, and one of 
> them might
> erroneously perform some operation on one of them. Obviously there could also 
> be a
> security issue with this, depending on what your application does.
>
> Basically, when a descriptor is "handed off" to a child process by forking, 
> you
> generally want to close it in the parent, unless it's used for parent-child
> communication. (There are some cases where the parent wants to keep it open 
> for
> some reason, but they're rare.)
>
> On a similar note, if you exec a different program in the child process (I 
> wasn't
> sure from your description), it's a good idea for the parent to set the 
> FD_CLOEXEC
> option (with fcntl) on its listening socket and any other descriptors that 
> shouldn't
> be passed along to child processes. You could close these manually in the 
> child
> process between the fork and exec, but FD_CLOEXEC is often easier to maintain.
>
> For some applications, you might just dup2 the socket over descriptor 0 or
> descriptor 3, depending on whether the child needs access to stdio, and then 
> close
> everything higher.
>
> Closing descriptors not needed by the child process is a good idea even if you
> don't exec, since it can prevent various problems and vulnerabilities that 
> result
> from certain classes of bugs. It's a defensive measure.
>
> The best source for this sort of recommendation, in my opinion, remains W. 
> Richard
> Stevens' /Advanced Programming in the UNIX Environment/. The book is old, and 
> Linux
> isn't UNIX, but I don't know of any better explanation of how and why to do 
> things
> in a UNIX-like OS.
>
> And my favorite source of TCP/IP information is Stevens' /TCP/IP Illustrated/.
>
> > May it explain my problem?
>
> In this case, I don't offhand see how it does, but I may be overlooking 
> something.
>
> > I suppose that, if for some reason, the communication with the client is 
> > lost
> > (crash of client, loss of network, etc.) and keepalive is not enabled, this 
> > may
> > fully explain my problem ?
>
> It would give you those symptoms, yes.
>
> > If yes, do you have an idea of why keepalive is not enabled?
>
> The Host Requirements RFC mandates that it be disabled by default. I think the
> primary reasoning for that was to avoid re-establishing virtual circuits (e.g.
> dial-up connections) for long-running connections that had long idle periods.
>
> Linux may well have a kernel tunable or similar to enable TCP keepalive by
> default, but it seems to be switched off on your system. You'd have to consult
> the documentation for your distribution, I think.
>
> By default (again per the Host Requirements RFC), it takes quite a long time 
> for
> TCP keepalive to detect a broken connection. It doesn't start probing until 
> the
> connection has been idle for 2 hours, and then you have to wait for the TCP
> retransmit timer times the retransmit count to be exhausted - typically over 
> 10
> minutes. Again, some OSes let you change these defaults, and some let you 
> change
> them on an individual connection.
>
> --
> Michael Wojcik
>