Hi Chris,
first thanks for these real world tests.
Conclusion
We have a problem with the data exchange functions for certificates. It looks to me like the time taken to export certificates from the CA is in proportion to the current volume of the CA (other test results show that a CA of 8000 certificates takes 39 minutes to export 1000, a CA of 9000 certs takes 58 minutes to export 1000 and 10,000 takes 67 minutes to export 1000).
The reason is simple and inacceptable for high volume systems:
export-import.lib --> sub exportObjects --> while ($object)
This while loop iterates over all (!!!) existing certificates. The best idea to solve this would be a database table with the following stuff:
objecttype status serial(int) key(char) module_id export_status CERTIFICATE valid 1005 - 12 1
The select query would be really complex:
table certificate == c table dataexchange == d
select c.serial from c, d where
not (d.module_id=3 and
d.objecttype=certificate and
d.status=valid and
d.export_status=1 and
d.serial=c.serial) or not (für alle anderen module_ids)This is actually impossible because we cannot(?) remove the DBM support from 0.9.2 shortly before the release.
The major problem in the background is that we support dataexchange with one root node and several children. So the upload is trivial but the enrollment is horrible because you must check the dependencies for all "client" nodes. So the export performance is linear with the number of client "nodes". We can only speedup this problem by performing the dataexchange always between explicit pairs of nodes. This would reduce the query to
select c.serial from c, d where
not (d.module_id=3 and
d.objecttype=certificate and
d.status=valid and
d.export_status=1 and
d.serial=c.serial)This would be much faster than the actual solution where we perform the merge in perl. Nevertheless if you use such an architecture than it is not important when you wait (120 minutes or 6 times 20 minutes makes no difference).
Summary: We lose the most performance today during performing SQL stuff in Perl. There is no way (because of DBM) to send direct SQL statements to the database backend.
Finally we have two options:
1. Release 0.9.2 very soon (within days or weeks) and then remove the DBM support.
2. Delay 0.9.2 and remove DBM support before the release
I would prefer the first version. There are no open issues. The LDAP stuff has still some problems but it is much more advanced than the one in 0.9.1 and LDAP stuff is backportable. There are many new issues which all require big changes in OpenCA's backend (key management, dataexchange issue, auditing). So I think it's time to release or we start destabilizing OpenCA before releasing it.
Any ideas?
Michael -- _______________________________________________________________
Michael Bell Humboldt-Universitaet zu Berlin
Tel.: +49 (0)30-2093 2482 ZE Computer- und Medienservice Fax: +49 (0)30-2093 2704 Unter den Linden 6 [EMAIL PROTECTED] D-10099 Berlin _______________________________________________________________
------------------------------------------------------- This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 Project Admins to receive an Apple iPod Mini FREE for your judgement on who ports your project to Linux PPC the best. Sponsored by IBM. Deadline: Sept. 13. Go here: http://sf.net/ppc_contest.php _______________________________________________ OpenCA-Devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/openca-devel
