Thank you very much for your detailed and very helpful response. I have lot on my plate to work on. I will get back to the list if I have anymore questions. -Anand
-----Original Message----- From: owner-openssl-us...@openssl.org [mailto:owner-openssl-us...@openssl.org] On Behalf Of Ger Hobbelt Sent: Wednesday, October 07, 2009 7:20 PM To: openssl-users@openssl.org Subject: Re: Creating Encryption/Decryption Filter C++ On Tue, Oct 6, 2009 at 10:51 PM, Patel, Anand <anand.pa...@cobham.com> wrote: > I would like to know how can I implement general purpose > encryption/decryption filter that can be used with BIO objects. > > Basically, filter should get the data before it is written out to the > stream/socket/memory. This allows my filter to encrypt/decrypt the data > for any kind of source/sink object. > > Furthermore, is it possible to implement devices also, if so how can I > do that. Before I knew BIO concept I had implemented device and filter > using BOOST::Iostream libraries. > > BIO seems very attractive concept so far, but I don not know how to > attach my version of device and filter to it. > > Thank you, > -Anand Your question is a bit ambiguous to me; the first part seems to request how to use BIOs as general purpose encryption/decryption data filters while the second seems to request how one code BIOs of their own, such as custom filters and devices (source/sinks in OpenSSL BIO parlance). The first is the shortest and easiest answer: use a BIO_f_cipher() BIO and set it up with the cipher you require. BIO_f_cipher wraps the EVP routines so anything that can be done through them can be done through BIO_f_cipher(). There's also a BIO_f_md() filter for generic secure hashing of your data, but you should take care to read it's documentation page before using as you'll need to know how to extract the calculated hash. You can combine such BIO filters in a chain with any BIO source/sink device at the end; often used types are supported in OpenSSL off the shelf, i.e. BIO_s_fd() for 'file handle' based file I/O, BIO_s_socket() for (TCP) socket-based I/O, BIO_s_mem() for special purpose memory-buffered I/O (BIO_s_mem() has a few caveats, see also second part below. In usage, such can be overcome by using a bidirectional buffering device instead, called a 'BIO pair': see BIO_s_bio()) As BIOs are essentially 'C' structures and callbacks invoked through OpenSSL API functions, such can be simply used in C++ classes as well. Use the OpenSSL provided APIs to create them, link them in a chain (BIO_push()), access them (BIO_read(), etc.) and clean them up (BIO_close(), BIO_free()). Then there's the answer to the second part: how to create your own BIOs... very doable, but it requires a bit more studying of existing code and documentation. For generic encrypt/decrypt BIOs see the sources crypto/evp/bio_enc.* and crypto/evp/bio_ok.* we use the BIO_f_cipher() one around here most of the time -- that would be the one in bio_enc.c; we have an edited BIO_f_reliable() over here so can't say anything about that one in the official distro. The BIO system is indeed very attractive indeed, and we use it a /lot/ around here, but a few caveats apply: - the BIO system is a 'C' design. Which means you'll need some extra handywork to have it behave like C++ objects or include C++ objects in a BIO filter chain. - some quirks in very advanced stuff regarding detaching/attaching chains and pushing control messages; I should find/make time to prep and submit to rt@ :-( Don't let that bother you, it won't matter unless you try some quite fancy stuff. As the BIO system is 'C' based, your own, custom BIO filters and/or devices should have a 'C' interface. That is: you /can/ code BIO's in C++, but you'll need to supply an extern 'C' facade with those to make it behave. Your BIO structure (as visible to 'C') should then be made to carry the C++ 'this' pointer around so you can properly invoke your C++ virtual and non-virtual class methods from the 'extern "C"' facade functions and callbacks that should come with your BIO filter / device. On the other hand, you can code them entirely in 'C' (a C++ compiler can generally also compile 'C' code after all); apart from a few quite advanced filters we took that approach over here. I find that reading the crypto/BIO/* source code files is most helpful (but then I don't have a problem grokking code); take heed and note that a few of these are a little light on the error/failure checking and handling part (I'm rather a*** retentive in that regard ;-) ) For a filter example, look how evp/bio_enc.c was done, then compare with a different, slightly more complex filter, such as the BIO_f_buffer() bidirectional buffering filter found in crypto/BIO/bf_buff.c Note that some filters are unidirectional by design/implementation, despite the fact that the basic BIO structure design facilitates bidirectional I/O. For clarity the definition of 'bidirectional I/O capable' as I use it: A filter or device is 'bidirectional I/O capable' when read(I) and write(O) data transmissions do not influence one another within the BIO implementation. [Mark the use of the word 'within' here] In other words: IN and OUT data directions are independent of one another as far as the BIO is concerned.' 'UNIdirectional' filters include BIO_f_base64(), which won't play nice when you pump partial data snippets through it in both directions at the same time: both ends will receive garbled content then. Such filters (and devices: see BIO_s_mem()) are easily recognizable in their implementation as read and write direction methods share common buffer(s) and/or state (BIO-attached flags and counters which are thus persisted beyond the callback invocation). I mention this so you are warned and don't copy&paste an arbitrary filter source when you need to create a bidirectional I/O capable BIO filter. Examples of various levels of 'bidirectional I/O support' in existing BIO filters and devices: - BIO_f_buffer() (crypto/bio/bf_buff.c) is bidirectional I/O capable (notice the separate buffers and state for read and write direction) - BIO_f_linebuffer() (crypto/bio/bf_lbuf.c) is a bidirectional I/O capable filter which only 'works' in one direction: the linebuffering is only performed in /write/ direction. Yet it is 100% bidirectional I/O capable. - BIO_f_base64() (crypto/evp/bio_b64.c) is a /non/-bidirectional filter: notice how the read and write methods both meddle with the single 'shared' base64<->raw text buffer. It only 'works' as expected when you feed it entire messages, explicitly flush the buffer and only then switch data direction (I<->O) in the filter chain. Wicked surprises lie this way, especially when mixing this kind of filter with implicitly bidirectional usage filters such as, e.g., the BIO_f_ssl() SSL protocol filter in the same chain. - BIO_s_socket() (crypto/BIO/bss_sock.c) is fully bidirectional I/O capable: read and write activity passes to the operating system without the one influencing the other within the BIO. - BIO_s_mem() (crypto/BIO/bss_mem.c) is not bidirectional I/O capable as per the definition above; it /can/ be used to reverse data direction from write to read, as the writes will fill the shared buffer and the read calls will extract data from that same buffer, but it has a few peculiarities that don't make it a good example to base your own BIOs on, in my opinion. For a device, the simplest is of course the NULL BIO device BIO_s_null() (which acts as the any-platform BIO equivalent of UNIX /dev/null) in crypto/BIO/bss_null.c and maybe you'ld have a look at the file-handle based source/sink BIO BIO_s_fd source code too to get an idea about how, what, goes where for a source/sink ('device') BIO. crypto/BIO/bss_fd.c Note that in BIO land, a 'device' is called a source/sink. Also note that the BIO structure and callback set must be constructed for each BIO: yes, a BIO is defined by the configuration structure and the ID plus function callbacks listed in there, as returned by the various BIO_f_xyz (for filter xyz) and BIO_s_xyz (for source/sink device xyz) functions. These above two parts must be accessible in 'C', i.e. must be done as 'extern "C"' code pieces. Advise: make sure your BIO has a unique identifier for its BIO_TYPE_xyz, so you can use the BIO chain traversal and BIO lookup functions provided by OpenSSL as they are. There is no problem 'attaching' a C++ object instance to such a BIO, as each BIO comes with it's own internal structure, where the 'this' pointer of the C++ instance can be placed inside the BIO instance. A very nice example and a hint how one could do this can be found by having a look at the BIO_s_file() source/sink, i.e. the BIO which represents a 'FILE*' pointer (crypto/BIO/bss_file.c). Here, note once more that each BIO instance is allocated on the heap through a call to BIO_new() and the BIO-defined allocator callback. Note that any 'custom data' which is specific to the BIO can be allocated any which way you like -- as long as you clean up accordingly afterwards in your own 'free' callback! and put a pointer to this custom data in BIO->data for best practices. See in bss_file.c how the external/internal 'FILE*' pointer is attached to the fresh BIO instance. The hint here is that attaching a C++ class instance can be done quite similarly, by doing with its 'this' pointer what bss_file.c does with a 'FILE*' pointer. Lastly note that your BIO (filter or device) is completely free to define and allocate any custom data it deems necessary (such as a C++ class instance of type X); anything you allocate must be freed from the heap using the matching heap management function: malloc() vs. free(), new and new[] vs. delete and delete[] respectively. Of course all BIOs are cleaned up themselves by invoking BIO_free() -- or it's chain-oriented big brother BIO_free_all() - both of which will call your BIOs 'free' callback when the time comes. OpenSSL chains and manages BIOs, it does not care what is inside them: your callbacks have to provide for that. -- Met vriendelijke groeten / Best regards, Ger Hobbelt -------------------------------------------------- web: http://www.hobbelt.com/ http://www.hebbut.net/ mail: g...@hobbelt.com mobile: +31-6-11 120 978 -------------------------------------------------- ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
