Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/18/2013 08:39 PM, Rob Crittenden wrote: Rob Crittenden wrote: Petr Viktorin wrote: On 02/15/2013 10:43 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 07:23 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. This gets rid of most of the failures, but it still fails the certs for this IPA server/short name tests if the cert from ./make-cert is present (creating it is part of `make test`). If make-cert is run more times, it'll revoke the previous cert, so the test for revocation reason 4 fails as well. It looks like when using sizelimit Dogtag will always discard *newer* certs, ones with higher serials. Is it documented behavior or does Dogtag just happen to do that? It isn't documented anywhere I could find, it is just what dogtag returns I wonder how other people run their tests. This solution looks like it could break easily if people do something differently :( I'm not sure how to solve this properly. Perhaps not using Declarative, and checking by hand that the wanted certs are in the response and the unwanted ones are not, would work better. I ended up switching the test class. It is not a very fine-grained set of tests, mostly searching with limits and verifying that we fall within a reasonable range, but it is better than nothing. rob This works much better, thanks! Just two nitpicks now. The patch doesn't apply well, there's a conflict in VERSION and some added trailing whitespace, AFAIK this would be the only (first?) test that relies on Nose's ordering of test modules -- tests 0011 and 0030 rely on the other cert tests running first. Please at least mention that in a comment. Or better, move class test_cert_find to test_cert.py Rebased the patch and removed whitespace. I went ahead and combined this with the existing test_cert file. Originally test_cert was only tested against lite-server but since it works against a live dogtag server too it makes sense to combine them. I improved the set up documentation a little bit and tried to handle all the different configurations that one might see so that this should be runnable against either a live server or the lite-server for both the selfsign and dogtag backends. This relies on the user configuring ~/.ipa/default.conf to match the remote server. There is no way from a client to know what kind of CA backend a server is running. rob And the patch. rob ACK, thank you! -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Viktorin wrote: On 02/18/2013 08:39 PM, Rob Crittenden wrote: Rob Crittenden wrote: Petr Viktorin wrote: On 02/15/2013 10:43 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 07:23 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. This gets rid of most of the failures, but it still fails the certs for this IPA server/short name tests if the cert from ./make-cert is present (creating it is part of `make test`). If make-cert is run more times, it'll revoke the previous cert, so the test for revocation reason 4 fails as well. It looks like when using sizelimit Dogtag will always discard *newer* certs, ones with higher serials. Is it documented behavior or does Dogtag just happen to do that? It isn't documented anywhere I could find, it is just what dogtag returns I wonder how other people run their tests. This solution looks like it could break easily if people do something differently :( I'm not sure how to solve this properly. Perhaps not using Declarative, and checking by hand that the wanted certs are in the response and the unwanted ones are not, would work better. I ended up switching the test class. It is not a very fine-grained set of tests, mostly searching with limits and verifying that we fall within a reasonable range, but it is better than nothing. rob This works much better, thanks! Just two nitpicks now. The patch doesn't apply well, there's a conflict in VERSION and some added trailing whitespace, AFAIK this would be the only (first?) test that relies on Nose's ordering of test modules -- tests 0011 and 0030 rely on the other cert tests running first. Please at least mention that in a comment. Or better, move class test_cert_find to test_cert.py Rebased the patch and removed whitespace. I went ahead and combined this with the existing test_cert file. Originally test_cert was only tested against lite-server but since it works against a live dogtag server too it makes sense to combine them. I improved the set up documentation a little bit and tried to handle all the different configurations that one might see so that this should be runnable against either a live server or the lite-server for both the selfsign and dogtag backends. This relies on the user configuring ~/.ipa/default.conf to match the remote server. There is no way from a client to know what kind of CA backend a server is running. rob And the patch. rob ACK, thank you! pushed to master and ipa-3-1 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Viktorin wrote: On 02/15/2013 10:43 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 07:23 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. This gets rid of most of the failures, but it still fails the certs for this IPA server/short name tests if the cert from ./make-cert is present (creating it is part of `make test`). If make-cert is run more times, it'll revoke the previous cert, so the test for revocation reason 4 fails as well. It looks like when using sizelimit Dogtag will always discard *newer* certs, ones with higher serials. Is it documented behavior or does Dogtag just happen to do that? It isn't documented anywhere I could find, it is just what dogtag returns I wonder how other people run their tests. This solution looks like it could break easily if people do something differently :( I'm not sure how to solve this properly. Perhaps not using Declarative, and checking by hand that the wanted certs are in the response and the unwanted ones are not, would work better. I ended up switching the test class. It is not a very fine-grained set of tests, mostly searching with limits and verifying that we fall within a reasonable range, but it is better than nothing. rob This works much better, thanks! Just two nitpicks now. The patch doesn't apply well, there's a conflict in VERSION and some added trailing whitespace, AFAIK this would be the only (first?) test that relies on Nose's ordering of test modules -- tests 0011 and 0030 rely on the other cert tests running first. Please at least mention that in a comment. Or better, move class test_cert_find to test_cert.py Rebased the patch and removed whitespace. I went ahead and combined this with the existing test_cert file. Originally test_cert was only tested against lite-server but since it works against a live dogtag server too it makes sense to combine them. I improved the set up documentation a little bit and tried to handle all the different configurations that one might see so that this should be runnable against either a live server or the lite-server for both the selfsign and dogtag backends. This relies on the user configuring ~/.ipa/default.conf to match the remote server. There is no way from a client to know what kind of CA backend a server is running. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Viktorin wrote:
On 02/06/2013 07:23 PM, Rob Crittenden wrote:
Petr Viktorin wrote:
On 02/06/2013 12:44 AM, Rob Crittenden wrote:
This adds a cert-find command for the dogtag backend.
Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.
I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and
rely
on the selfsign backend by default.
rob
Thanks! The code works well, but I found a few issues.
These tests don't work when the full test suite is run: test_cert adds
and revokes additional certs that throw the code off.
Perhaps have the tests only query valid certs? I don't see that option
but I think it would be helpful to support.
I added some rather nasty hacks to the test to make things pass. I limit
the search to 10 certificates, which is the number start with by
default. There is an open dogtag ticket to return only VALID
certificates so we should be able to drop this eventually.
I had to go further on one of the revocation tests, limiting it to a
sizelimit of 1. The count changes every time the suite runs so this is
the safest for now. It also means that one test will fail if this is the
only part of the suite executed.
This gets rid of most of the failures, but it still fails the certs for
this IPA server/short name tests if the cert from ./make-cert is
present (creating it is part of `make test`).
If make-cert is run more times, it'll revoke the previous cert, so the
test for revocation reason 4 fails as well.
It looks like when using sizelimit Dogtag will always discard *newer*
certs, ones with higher serials. Is it documented behavior or does
Dogtag just happen to do that?
It isn't documented anywhere I could find, it is just what dogtag returns
I wonder how other people run their tests. This solution looks like it
could break easily if people do something differently :(
I'm not sure how to solve this properly. Perhaps not using Declarative,
and checking by hand that the wanted certs are in the response and the
unwanted ones are not, would work better.
I ended up switching the test class. It is not a very fine-grained set
of tests, mostly searching with limits and verifying that we fall within
a reasonable range, but it is better than nothing.
rob
From e7972a96abc62d41a90f9bd3e816c827627b7120 Mon Sep 17 00:00:00 2001
From: Rob Crittenden rcrit...@redhat.com
Date: Thu, 15 Nov 2012 10:55:33 -0500
Subject: [PATCH] Implement the cert-find command for the dogtag CA backend.
Use a new RESTful API provided by dogtag 10+. Construct an XML document
representing the search request. The output is limited to whatever dogtag
sends us, there is no way to request additional attributes other than
to read each certificate individually.
dogtag uses a boolean for each search term to indicate that it is used.
Presense of the search item is not enough, both need to be set.
The search operation is unauthenticated
Design page: http://freeipa.org/page/V3/Cert_find
https://fedorahosted.org/freeipa/ticket/2528
---
API.txt | 23 +++
VERSION | 2 +-
ipalib/plugins/cert.py | 137 -
ipaserver/plugins/dogtag.py | 138 +
ipaserver/plugins/rabase.py | 8 +
tests/test_xmlrpc/test_cert_find.py | 289
6 files changed, 594 insertions(+), 3 deletions(-)
create mode 100644 tests/test_xmlrpc/test_cert_find.py
diff --git a/API.txt b/API.txt
index 8fbfe6f5d8da44e991b8d1a36725fc6ace1f0616..6b997f37b455366c66b34fd2df11c2acaa79d739 100644
--- a/API.txt
+++ b/API.txt
@@ -425,6 +425,29 @@ args: 1,0,2
arg: Any('methods*')
output: Output('count', type 'int', None)
output: Output('results', (type 'list', type 'tuple'), None)
+command: cert_find
+args: 0,17,4
+option: Flag('all', autofill=True, cli_name='all', default=False, exclude='webui')
+option: Flag('exactly?', autofill=True, default=False)
+option: Str('issuedon_from?', autofill=False)
+option: Str('issuedon_to?', autofill=False)
+option: Int('max_serial_number?', autofill=False, maxvalue=2147483647)
+option: Int('min_serial_number?', autofill=False, minvalue=0)
+option: Flag('raw', autofill=True, cli_name='raw', default=False, exclude='webui')
+option: Int('revocation_reason?', autofill=False, maxvalue=10, minvalue=0)
+option: Str('revokedon_from?', autofill=False)
+option: Str('revokedon_to?', autofill=False)
+option: Int('sizelimit?', default=100, minvalue=0)
+option: Str('subject?', autofill=False)
+option: Str('validnotafter_from?', autofill=False)
+option: Str('validnotafter_to?', autofill=False)
+option: Str('validnotbefore_from?', autofill=False)
+option: Str('validnotbefore_to?', autofill=False)
+option: Str('version?', exclude='webui')
+output: Output('count', type 'int', None)
+output: ListOfEntries('result', (type 'list', type 'tuple'),
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/07/2013 03:08 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 I'm going to open a UI ticket once the API is finalized. I didn't want to give you a moving target to work against. rob I see that the search requires to specify options for attributes to search on. There is no general CRITERIA positional argument as in other find commands or am I mistaken? Is it possible to add the CRITERIA argument? Is the no 'OR' search an obstacle for it? If so we would really need to push the ticket #191 because UI doesn't support search by only specifying specific attributes yet. -- Petr Vobornik ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Vobornik wrote: On 02/07/2013 03:08 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 I'm going to open a UI ticket once the API is finalized. I didn't want to give you a moving target to work against. rob I see that the search requires to specify options for attributes to search on. There is no general CRITERIA positional argument as in other find commands or am I mistaken? Is it possible to add the CRITERIA argument? Is the no 'OR' search an obstacle for it? If so we would really need to push the ticket #191 because UI doesn't support search by only specifying specific attributes yet. Your analysis is correct. It may be considered a hack but what if I treat subject as the CRITERIA argument? rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/14/2013 03:34 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/07/2013 03:08 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 I'm going to open a UI ticket once the API is finalized. I didn't want to give you a moving target to work against. rob I see that the search requires to specify options for attributes to search on. There is no general CRITERIA positional argument as in other find commands or am I mistaken? Is it possible to add the CRITERIA argument? Is the no 'OR' search an obstacle for it? If so we would really need to push the ticket #191 because UI doesn't support search by only specifying specific attributes yet. Your analysis is correct. It may be considered a hack but what if I treat subject as the CRITERIA argument? rob Better that than nothing. Just a confirmation: when user does not set any option, it will return all certificates? Or it will return nothing? I see Web UI implementation this way: 1) implement simple search with the hack now 2) if there will be time before release (after the refactoring and other tickets) implement #191 (will require UXD input) to implement this the proper way. We can move #191 to triage to decide it. -- Petr Vobornik ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/06/2013 07:23 PM, Rob Crittenden wrote:
Petr Viktorin wrote:
On 02/06/2013 12:44 AM, Rob Crittenden wrote:
This adds a cert-find command for the dogtag backend.
Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.
I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and rely
on the selfsign backend by default.
rob
Thanks! The code works well, but I found a few issues.
These tests don't work when the full test suite is run: test_cert adds
and revokes additional certs that throw the code off.
Perhaps have the tests only query valid certs? I don't see that option
but I think it would be helpful to support.
I added some rather nasty hacks to the test to make things pass. I limit
the search to 10 certificates, which is the number start with by
default. There is an open dogtag ticket to return only VALID
certificates so we should be able to drop this eventually.
I had to go further on one of the revocation tests, limiting it to a
sizelimit of 1. The count changes every time the suite runs so this is
the safest for now. It also means that one test will fail if this is the
only part of the suite executed.
This gets rid of most of the failures, but it still fails the certs for
this IPA server/short name tests if the cert from ./make-cert is
present (creating it is part of `make test`).
If make-cert is run more times, it'll revoke the previous cert, so the
test for revocation reason 4 fails as well.
It looks like when using sizelimit Dogtag will always discard *newer*
certs, ones with higher serials. Is it documented behavior or does
Dogtag just happen to do that?
I wonder how other people run their tests. This solution looks like it
could break easily if people do something differently :(
I'm not sure how to solve this properly. Perhaps not using Declarative,
and checking by hand that the wanted certs are in the response and the
unwanted ones are not, would work better.
The API.txt check fails:
Option sizelimit? of command cert_find in ipalib, not in API file:
Int('sizelimit?', default=100, minvalue=0)
Ouch. I thought I had fixed that, obviously not. Done now.
What are --all and --raw for? Is the plan to implement --all if/when
Dogtag supports requesting additional data?
Correct, they don't do anything at the moment. I have an RFE open to
return additional data from certs. Once that is done then all will make
sense. I don't know that raw will ever do anything interesting here but
it comes with all commands.
The format of --validnotbefore-to and friends should be mentioned in
--help text; the following is confusing:
$ ipa cert-show 1
[...]
Not Before: Wed Feb 06 09:32:17 2013 UTC
[...]
$ ipa cert-find -h
[...]
--validnotbefore-to=STR
Valid not before to this date
[...]
$ ipa cert-find --validnotbefore-to='Wed Feb 06 09:32:17 2013 UTC'
ipa: ERROR: invalid 'validnotbefore_to': time data u'Wed Feb 06 09:32:17
2013 UTC' does not match format '%Y-%m-%d'
It was listed in the top block but I added it to the usage help as well
for clarity.
Could you make the help text for --exactly more specific?
Done.
Please remove the extra whitespace at the end of dogtag.py
I'd welcome a link to the design page in the commit message.
both done
rob
--
Petr³
___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Jan Cholasta wrote: Hi, On 6.2.2013 00:44, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob I have one design question: why do you emulate object interface with Command plugins? Wouldn't it be better to add an actual Object plugin and Method plugins? That way you would not have to duplicate the Object bits for certs and as a result, the code would be cleaner and consistent with the rest of our plugins. Honza I forget the details of the reasoning but IIRC it is because these commands aren't backed by LDAP. So the normal things we get out of Object don't apply. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 -- Petr Vobornik ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Vobornik wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 I'm going to open a UI ticket once the API is finalized. I didn't want to give you a moving target to work against. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Hi, On 6.2.2013 00:44, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob I have one design question: why do you emulate object interface with Command plugins? Wouldn't it be better to add an actual Object plugin and Method plugins? That way you would not have to duplicate the Object bits for certs and as a result, the code would be cleaner and consistent with the rest of our plugins. Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
